Jonathan Berman is Associate Professor of Marketing at the London Business School. His main research focus is on judgment and decision-making.
In this conversation, we talk predominately about Jonathan's paper "Moral Choice When Harming is Unavoidable" that came out this year in Psychological Science. As part of our conversation about this paper, we also discuss open science practices, especially preregistration.
BJKS Podcast is a podcast about neuroscience, psychology, and anything vaguely related, hosted by Benjamin James Kuper-Smith. New conversations every other Friday. You can find the podcast on all major podcasting platforms (e.g., Spotify, Apple/Google Podcasts, etc.).
Timestamps
0:00:08: How Jonathan got into the field he currently works in
0:18:55: Discussing Jonathan's paper "Moral choice when harming is unavoidable"
0:35:03: Framing of moral decisions
0:42:43: Which studies to include in a paper?
0:48:46: Simple experiments
0:55:39: How Jonathan's research fits into a marketing department
1:02:33: Open science
1:09:58: File drawers and preregistration (with additional contributions from Jonathan's 6-month old child)
Podcast links
Website: https://bjks.buzzsprout.com/
Twitter: https://twitter.com/BjksPodcast
Jonathan's links
Website: www.london.edu/faculty-and-research/faculty-profiles/b/berman-j-z-1
Google Scholar: https://scholar.google.de/citations?user=iIMbl9QAAAAJ&hl=de&oi=ao
Twitter: https://twitter.com/jberman81
Ben's links
Website: www.bjks.page/
Google Scholar: https://scholar.google.co.uk/citations?user=-nWNfvcAAAAJ
References
Berman, J. Z., & Kupor, D. (2020). Moral choice when harming is unavoidable. Psychological Science.
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science.
-
[This is an automated transcript that contains many errors]
Benjamin James Kuper-Smith: [00:00:00] So I was looking at your CV earlier after your bachelor's, you worked at Miramax and at, um, literary agent.
Jonathan Berman: That's right.
Benjamin James Kuper-Smith: Um, that sounds, uh, well, that's not something I expected from an academic economist necessarily. Uh, can you, so I'm just curious, like, did you plan on becoming a research economist or,
Jonathan Berman: uh, no.
Um, so I can tell you kinda the story about how I got into the field I am in now. Um, well, when I was in, when I was in undergrad, I didn't really know what I wanted to do. I thought that if I was gonna do a PhD, I thought about maybe economics and maybe philosophy, but I was never, and I studied a little bit of philosophy, a little bit more economics, but I was never.
So into it. And, um, [00:01:00] uh, I, I didn't really have the self-confidence to who pushed that forward, but I was a, a, a big, I've always been a big, um, at least in my youth, I was a really big fiction reader. Um, and I was even more so into Phil. And so when I graduated from undergrad, I said to myself, well, this is the time in my life where, um, you know, I could try things and see if it is right for me.
I had my eye on, on, uh, being a literary agent because that seemed more like the business side of the publishing industry, and that seemed like something that I would be well suited for, right? Um, and so I wanted to get that, but I couldn't really get a job right off the bat. But I, I was able to get a job at Miramax first.
Um, and so that was really my first job [00:02:00] out of undergrad. And, um, I was actually on the marketing and I was in the distribution side. Um, as you could imagine, it was a dysfunctional environment, uh, for bullying and, uh, uh, kind of, uh. It, its reputation for being, um, at least from the higher ups as, as being dysfunctional people and being bullies, uh, and being, you know, um, uh, not in a, a warm and encouraging environment was true.
Uh, so, um, but, but after, uh, after I left there, I was able to find a job at a literary agency. Um, I shortly found out that that wasn't the right job for me, but it was there when I started reading books about decision making
Benjamin James Kuper-Smith: as
Jonathan Berman: part of your
Benjamin James Kuper-Smith: job or just, uh, on the side?
Jonathan Berman: On the side, on the side. So everybody books about decision making, [00:03:00] and I didn't know that people studied decision making in the same way that these books and if we're talking about it, and that's when it all clicked, so to speak.
I said to myself that I, I don't know that much about this topic, but. This speaks to me more than anything else that I've encountered in academia or from an intellectual perspective. I need to study it more. And that's when I applied for a master's degree at the London School of Economics in decision sciences.
And that was really the beginning of everything for me. Uh,
Benjamin James Kuper-Smith: and you hadn't encountered these kind of books in your, I I would imagine, like if you study economics and philosophy that uh, there would be some overlap with the kind of decision making literature or,
Jonathan Berman: well,
Benjamin James Kuper-Smith: or
Jonathan Berman: not when I was, yeah. When I was studying economics, for some reason I was a little bit more drawn to macroeconomics.
For instance, I, uh, you know, ma [00:04:00] microeconomics, which is much more aligned with what I study now, uh, came pretty easily to me. But I was, was more intrigued by macroeconomics. I was more intrigued by understanding the entire system of the economy as a whole. Um, similarly from a philosophy standpoint, I mean, I study kind of ethics now.
I wasn't as interested in ethics in undergrad. I was more interested in existentialism and, uh, a lot of these more abstract, you know, metaphysics, those types of concepts. But I found the material really, really challenging, you know, and it didn't. It wasn't something that, that, that really fit well with the way I think about the world.
Uh, I found it interesting, but a lot of the times I found it interesting because I didn't quite understand it and I thought to myself, you know, there's some knowledge here that I don't have access to that seems like it's revealing deep truths about the world. [00:05:00] Um, and that piqued my interest. Um, so I, in many ways, I was drawn to things I didn't quite understand, which is good and bad.
You know, it's, it's, um, uh, you're curious about those topics. You kind of wanna dig deeper, but it didn't really necessarily fit with, um, with ways that. I think that you, uh, well fit with the way I think about the world. So when I, where I did undergrad, they didn't have a big, for instance, analytic philosophy department.
It was all, there's this analytic and continental divide. Continental philosophy. Yeah. Can
Benjamin James Kuper-Smith: you, what exactly is that D divide?
Jonathan Berman: So, continental philosophy is, um, uh, I mean, it, it dates back to kind of analytic. Was, is more, I would say my, my understanding of it, analytic is a much more, kind of a logic based is much more, uh, trying to understand kind of, uh, principles, building arguments in a [00:06:00] very structured way where continental philosophy, um, is, is more, um, uh, you know, it, it typically, it gets associated with like postmodernism.
Um, which is more, you know, the writers are a lot more, um. Uh, I, I'm not quite sure kind of how to describe it, but it's a lot more, um, I, I don't wanna say fluffy, but that's the, it's not because it's not fluffy at all. Um, it's a lot. I guess you could argue it's a lot more's more qualitative rather than quantitative.
Mm-hmm. Um, if that, that's the way I could describe it. There are gonna be, other people will do better jobs describing that dis distinction. Uh, but I think that that's, um, you know, that's as good as I'm gonna get off the cuff. [00:07:00] So, like, you know, uh, if you've heard of. Uh, like Fko, I would say is more of a, um, you know, he's a big name.
I, I think he's a big name in more of this continental philosophy, uh, uh, deida, um, you know, a group of these, of these postmodern so type thinkers. Yeah. And I would've benefited a lot from a more analytic, uh, type of, of department in my undergrad. In some ways. I'm glad I didn't do that because I probably, I could have seen myself pursuing a PhD in philosophy, which I don't think I would've, uh, uh, done well in, so.
Benjamin James Kuper-Smith: I'm still, uh, uh, maybe last question about literary agency. Um, what exactly does the job of an assistant literary agent entail? Do you just get sent like 20 books a day and you have to decide which of those are worthy for further consideration? And also what, uh, kind of books were, was [00:08:00] your agency or you handling?
Jonathan Berman: Right. So different literary agents in different stages of their career, um, will have different ways that they manage their, uh, their agency. So I was working for the George Borchard literary Agency. George Borchard was, um, or is, um, um, he was, he, he was very well established when I joined. Agency. And so to give you an example, one of the first people he signed was Ellie Viel, um, who wrote kind of Knight and is famous for winning, I believe two noble prizes, one for peace and one for literature.
Um, and um, and George Bouchard, uh, is French and he moved to America and he kind of got his start representing a lot of French writers in America. Alright, so basically 95% of any French [00:09:00] writer you could think of that is big enough to be well, uh, uh, translated in the United States. He probably represented them and then he started representing his, his own people and he is in what they call kind of the literary, uh, uh, more the literary.
Um, fiction and literary non-fiction world. So what do we, what do I mean by that? These are what you can consider to be, um, they're not niche books and that it's not like he's doing cookbooks. It's not like he's doing pop science books, or it's not like he's doing self-help books. He's doing very literary, representing very literary auth literary authors, the type of authors who, you know, win, uh, Pulitzer Prizes, national Book Awards and so on.
So he had multiple, uh, um, you know, every year, you know, he may have a couple people nominated for one of [00:10:00] those, those prizes, small family organization. And I just had a variety of tasks that I would do. Sometimes I would read. He wasn't accepting, um, new manuscripts, so it wasn't like we were reading, you know.
20 books a month, but occasionally I would get one to read to see if it was, um, uh, uh, publishable. He wasn't accepting new clients. You know, the more the younger you are, the more you're accepting new clients, the more you're reading a lot. Um, rather, I, you know, I was, I did a little bit of the accounts. I did, for instance, uh, audio books, you know, I would try to negotiate those contracts, try to get those contracts set, um, and just manage the relationships.
Um, the literary agent is really kind of the go between, between the writer and the publisher. And so you manage those contracts, you manage the relationships, uh, very much. Uh, uh, uh, you know, in addition to trying to, dealing with all the money matters you're dealing with, the relationship matters
Benjamin James Kuper-Smith: as
Jonathan Berman: well.[00:11:00]
Benjamin James Kuper-Smith: But how it, it sounds to me like it was a, it's a very established, when you were there, it was a very established agency already. So how did you get that with. Well, I dunno what you did before, but it seems like you don't have anything that's particularly what I would assume is relevant work experience or, or did you?
Jonathan Berman: Right. So the one aspect of my background I left out is, yeah, because I knew I wanted to get into the literary world. I took this summer publishing course at Columbia University, kind of a, a short course that's kind of a primer for people who wanna get into the publishing industry. Mm-hmm. Um, and uh, uh, and then I just sent out, um.
Kind of my CV sent out letters to a variety of people, and luckily he, they needed someone and they responded and everything worked out quite well. You know, it was almost a stroke of luck that I sent them a letter, um, the time that they needed to hire someone. Uh,
Benjamin James Kuper-Smith: I see.
Jonathan Berman: [00:12:00] So, um, and I think because I did that course, there were, that, that was like a signal to, to at least interview me among the other resumes that they had been receiving.
Benjamin James Kuper-Smith: And I, I could imagine also that many resumes they would've been receiving would've been more of the literary kind where people studied English or something. But I guess
Jonathan Berman: right,
Benjamin James Kuper-Smith: from what you described, that is maybe not even what they needed that much, right. Because they weren't, they didn't have to assess lots of new books all the time.
Jonathan Berman: Right. I mean, I think that they, you know, having an English background suggests that you're passionate about
Benjamin James Kuper-Smith: Yeah.
Jonathan Berman: Uh, the literary world, um, and. Uh, you know, they also, I think, needed someone to do the accounts. And so having an econ background that was, uh, uh, something that I could do. Uh, it was a wonderful place to work, just for instance, because they were so kind of traditional, we, we used typewriters to type up all our correspondence.
And so, but this was
Benjamin James Kuper-Smith: in 2004 to two, 2006, just
Jonathan Berman: for
Benjamin James Kuper-Smith: context,
Jonathan Berman: right? Yeah, yeah, [00:13:00] yeah. Uh, back when no one was really using typewriters anymore, that was just like a pure joy. You know, you type up these correspondences and then you wrap up a manuscript, you know? Um, we weren't tying little ribbons on it, but it was almost like that.
And
Benjamin James Kuper-Smith: like sealing it with wax.
Jonathan Berman: Yeah, yeah, yeah, yeah. Um, um, so I had a, I had a great time there, uh, but. Uh, but it was also a good experience for me to realize that this is not, um, uh, I didn't have, I didn't have the, let's say, the client relationship skills at that time to, to go out on my own and say I'm gonna really start, uh, my own, uh, uh, literary agency or literary career.
Um, I always felt I was on the outside of the publishing world looking in, and you really need to kind of force your way in to start trying to get clients and represent them. Um, yeah, so, uh, uh, it was a great experience and it's a great experience because, [00:14:00] you know, it was one of these things that you, you learn not to do.
Um, you try it out and, and I try it out and I realized quickly that that wasn't a long-term career strategy for me.
Benjamin James Kuper-Smith: Yeah. I find that's in many cases, almost the most important thing to learn,
Jonathan Berman: right?
Benjamin James Kuper-Smith: Like just having those kind of restrictions, like, this is what I don't, what I'm not gonna do.
Jonathan Berman: Right, right.
Benjamin James Kuper-Smith: Um, yeah. But, uh, so then you kind of left just that kind of behind the literary scene, um, or you still reading lots of novels or,
Jonathan Berman: well, I'm still, I, I'm still a curious guy, you know, um, I've, as is, I think is, as I think, uh, I mean, I, my, my lay theory is that this is true, this is a general phenomena. As you get older, you get more interested in nonfiction.
Uh, I don't know if, I don't know how general that is, but that's my, my intuition. I still read a lot of fiction, but, and maybe because of what I do now, [00:15:00] um, I started reading a lot more nonfiction and, um. There was a period of time, and I think that this is true for a lot of academics. During your PhD, uh, during your young career, you just feel like you don't have time for books.
Uh, and, uh, you know, may as well read a paper. And a lot of times non-fiction books, the introduction or the first few chapters tells you the whole story, and you may as well read a very good paper. Some, um, I would argue that kind of many of the best papers I've read, uh, are more informative than many, many books I've read.
You know, so it just seems like a waste of time for a lot of people, but a number of years ago, and also you're constantly feeling behind in academia where you just don't feel like you have the time to sit down and read a book. Um, a number of years ago I decided, you know what? I have more time than I think I have.[00:16:00]
Um, I don't like feeling constantly behind. I need to make time. And I started reading more and more, and that has just been, in some sense, it's been useful to one, my self-confidence as an academic. That's something a lot of academic struggle with, uh, kind of imposter syndrome. One to kind of expand my horizons in topics that I find interesting, um, but didn't have that much, uh, knowledge about.
And it kind of helps you think about, um, well, reading more broadly has helped me think a lot more about, um, what are the big narratives that are important out there, or at least that are important and insightful to me. And what do I think is not being addressed by the current field. Um, I don't know how much that has directly impacted the things I actually study, but at least it, it has provided me with a lot of self-confidence.
Benjamin James Kuper-Smith: But how, self-confidence, just because you have a broader perspective or what, how does the [00:17:00] self-confidence play in there?
Jonathan Berman: Right. Just, uh, because I have a, I think it's because I have, uh, a deeper perspective on broad topics, you know? Um, uh, yeah, I, I, I, I feel like I'm, I feel like I think more about, about bigger and deeper topics than kind of the minutia of the given discipline that you find yourself in.
Um, and I have a tendency to kind of go into the minutiae and lose the big picture. I think that, you know, you study a problem long enough, you know, you see the details and that can lose track on what are the important big questions that you're evaluating, or how do we else, how do we think about, how do other people think about the world differently than.
They do in your sub-discipline, whatever that might be. Uh,
Benjamin James Kuper-Smith: yeah. Yeah. I mean, in some sense that's also part of why I'm doing this podcast, to give [00:18:00] people this kind of insight into different disciplines and how people Yeah. Yeah. For example, you as an economist, or I mean, like, you know, I work in, uh, large pro corporation social interactions, and that's a topic that, you know,
Jonathan Berman: okay,
Benjamin James Kuper-Smith: half the disciplines basically do at a university, right.
Um, whether, I mean, game theory Yeah. Whether it's biology, psychology, economics, computer science, whatever. Right. Um, I mean, then I feel like at some point that can also lead to a bit of an overwhelm because there's a sense of like, you have to learn all these different fields. Right. Um, but I, I do also feel that just.
Yeah, reading a, a paper from a different discipline, it's like evolutionary biology or something just gives you a completely different way of looking at what you're doing yourself.
Jonathan Berman: It's a balancing act and you have to find the balance that really works for you. Ultimately. That's the, uh, uh, that's what everyone kind of try, you know, I would think ought to try to do.
Um, so yeah.
Benjamin James Kuper-Smith: Anyway, so, uh, yeah, I wanted to, the reason I emailed you was [00:19:00]the specific reason is that I read, uh, this paper of yours, moral Choice when Harming is Unavoidable. So, shall we talk about that?
Jonathan Berman: I'd be happy to.
Benjamin James Kuper-Smith: Um, I think maybe I'm kind of half assuming that the people who listen to this will have.
Read this paper. Um, that's kind of the assumption, but we'll see. Um, I mean, I think I'm just gonna still read, I think two sentences
Jonathan Berman: Sure.
Benjamin James Kuper-Smith: From your general discussion that I think summarized the thing, uh, well, very well. That's probably where you wrote them. Um, so the first sentence is, um, we demonstrated that the preference to avoid inflicting any harm not only is distinct from, but also outweighs the preference to minimize its impact.
And then the last paragraph is, we found that decision makers who can completely avoid committing a harmful act. Frequently choose to do so. However, when committing some harm is unavoidable, decision makers become increasingly willing to trade off greater harm for greater benefits. [00:20:00] So
I think it's really interesting because it seems to me like you're kind of, what you found is that people have the almost two different modes of deciding how to think about cost benefit analysis or something like that.
Jonathan Berman: That's right. Just to kind of restate that, is that, or can I put it this another way?
People, uh, um, people really want to avoid committing a harmful act. Um, and when they're faced with a decision, should you, uh, commit some harm, which will then achieve some benefit or not? They say, I don't wanna do that. I don't wanna trade off harms for benefit. Uh, but then in situations where you. Both outcomes have some harm attached to it, then they're much more willing to say, well, let's conduct a, let's commit a little bit of harm, um, in order to achieve, uh, um, a great ordeal benefit.
Uh, and so kind of, uh, trade-offs [00:21:00] that they would, they would be very resistant to when they could completely avoid committing a harmful act. Uh, sometimes become quite desirable when, uh, when they cannot avoid committing harm and some harm. Must be committed.
Benjamin James Kuper-Smith: Mm-hmm. Yeah. And so, I mean, you understand this paper is, is a paper that shows in effect, right?
There's, um, no explanation of like why this exists. It's more like this effect exists. We've tested it in a few different contexts and, um,
Jonathan Berman: well we, we, yeah. We make the argument that really what this is all about is, um. Is very consistent with, with what some other researchers have showed in the past, is that people are very reluctant to, to commit a harmful act.
Okay. Uh, and then we argue that, that that is really what's driving this effect. So, uh, uh, this reluctant to, to, to commit a harmful act, um, is very [00:22:00] strong and people are just really hesitant to, to, to do that. Um, and so while people have looked at that angle, um, they haven't been able to kind of tease apart, well, is it that they're reluctant to commit a harmful act or they're reluctant to minimize harm and if they're, you know, so there's that kind of distinction that hasn't really been.
Uh, separated. People talk a lot about harm aversion, that people are harm averse. Um, but what does that actually mean? Does that mean that people wanna minimize the impact of the harm that is committed? Or do people wanna avoid, uh, harm entirely? Uh, you can't tease that apart when you're comparing doing some harm or doing no harm.
Um, and so we're able to tease that apart and we're able to say, well, that kind of, whatever, this notion of harm aversion really just means harm avoidance. You don't wanna commit a harmful act. [00:23:00] Um, and not that you want to minimize its impact. Uh, I feel like that that was a little bit of a confusing way to state, uh, the findings, but, uh, hopefully the point gets across.
Benjamin James Kuper-Smith: Yeah, I mean, maybe we can also so, uh, ask, I don't wanna get too much about the details of the study because I, I think one thing that's very nice about the study is that is. Fairly straightforward to understand. I think you have these five or six studies, they're all, um, binary choice for the, um, for the participants.
Um, but so I, I guess like one question I kind of have is in a more broader sense is that, so you have these different, like how do you decide which. Uh, I mean you, so you, okay, so you have this like, broader topic of doing harm, um, when sometimes you can't avoid to do so. Sometimes you can't, depending on what your options are.
Um, so how do you decide to choose like this specific like story context that you use in the different studies? So like, for example, the first one is [00:24:00] about putting nice support for someone who's terminally ill. The second one is about donating to a political party. You oppose, uh, and there's three most studies.
So like how exactly do you go about?
Jonathan Berman: Right. So we had a number of goals in mind. One is we wanted to use a range of different types of scenarios, um, to show that this is in particular about one type of decision that's being made. Um, and so we use. We use this range as much as we can to have a completely different types of scenarios.
Um, uh, so the, the classic scenario that is used to test these hypothesis is the trolley problem, kind of the trolley dilemma where, um, you could push someone over a foot bridge that will kill that person, but then would save multiple number of people who would otherwise be run over by a runaway trolley.
Or you choose not to push that person over the flip bridge, in which case you don't commit this harmful act, but you don't save the people as well. Um, [00:25:00] uh, there is, there has been, uh, some criticisms of the trolley dilemma that it is a little bit too, um, kind of abstract that people don't take it seriously.
Uh, so the other kind of classic. Dilemma that's been looked at. Um, and I believe, uh, Phil Tetlock, uh, as one of the first people to kind of look at this dilemma is this, uh, pulling life support for a terminated I child. So, uh, we adopted that as well. Um, and we used that dilemma. Uh, I will say as well that one of the big inspirations about this paper has been, uh, problems related to environmentalism.
Um, 'cause we talk about a lot about situations where people are f. Faced with decisions in which they cannot a avoid committing a harmful act. And I think that that is really true once we realize that so [00:26:00] many, uh, consumption options all involve committing some amount of harm. And so one example, um, I like to use is, uh, you know, food we eat, you know, you could order, uh, lunch option A or you could order lunch option B.
Well, regardless of what you're ordering, um, there are gonna be some carbon emissions associated with A or B. Now it may be much, much more, um, uh, carbon emissions associated with one option than the other one. But even kind of these low carbon food options, you order a plate of broccoli, it still requires carbon emissions.
To ship that, to package it, to sell it at the store, you know, um. A a And so a lot of the other scenarios we used, we wanted, you know, we wanted to use some sort of environmentally relevant decision. Uh, we didn't use that food example, but we used a lot about kind of cutting down acres with a rainforest.
[00:27:00] Um, uh, versus not, uh uh uh, so we used that as well. Uh, there were other scenarios that we used that didn't end up making it, uh, into the paper. Uh,
Benjamin James Kuper-Smith: why not?
Jonathan Berman: Uh, part of it was the review process. Uh, so we had a completely different type of scenario where we wanted to make sure we had one really incentive compatible design.
'cause a lot of these scenarios are very non, um. They're hypothetical. And so we wanted to use something that was more consequential. So initially we
Benjamin James Kuper-Smith: So you mean that there's real consequences
Jonathan Berman: make, there are real consequences
Benjamin James Kuper-Smith: make, yeah, exactly. Exactly. So like Charlie Dilemma, you're not gonna actually do that,
Jonathan Berman: right?
Benjamin James Kuper-Smith: Uh, but you can do it with monetary awards or something.
Jonathan Berman: That's right. Okay.
Benjamin James Kuper-Smith: That's right.
Jonathan Berman: And so initially we had a scenario where, uh, participants, uh, could either in one condition, either steal nothing from an anonymous partner that they've been [00:28:00] paired with or steal 1 cent from them. Okay. Um, uh, or we had a, and then we had another condition that says, simplifying the study, but it gets the point across.
Or they could steal 1 cent or 2 cents. Okay. They had to steal something. Right. Um, and, uh, the reviewers didn't like, kind of some of the language we were using. We, we used the word steal. They didn't, they didn't really like that. And so we had to have another in, we wanted to have another incentive compatible design.
Uh, and so we run this. During the election, and we thought, well, um, uh, maybe we could have people choose between, uh, donating money to different political parties, you know, with the idea that donating to the opposition party would be considered, uh, a harmful act. Um, and if you can't avoid donating to the opposition, would that change, uh, uh, how much you'd be willing to donate?
Benjamin James Kuper-Smith: Yeah. I like that you actually then incentivized that by saying, what was it? [00:29:00] Uh, this, yeah, this is actually one of, yeah. So you had this raffle and then if people in quota mark one, the raffle. Yeah, yeah, yeah. Then their, their donation to the opposing party would be made.
Jonathan Berman: Yeah.
Benjamin James Kuper-Smith: Um, yeah. I find it funny. So I, I looked at your, um, so one thing I also like about the studies that you have, it's very, um, open data and all that kind of stuff.
And, um, so I looked at the, uh, the, your data set that you have online and including com optional comments, I believe Yeah. By participants. Uh, and I think my favorite comment was from study two where someone commented, uh, Bernie 2020 America's only hope for real change, obviously in all caps. Um, I, yeah, and I think, uh, but what I wanted to get to with the comments was that one thing I found interesting is that I, I, I think this might have also been the case for study two, I can't remember, but I wrote it down for study.
Four, which is the, [00:30:00] your adaptation of the trolley dilemma, um, that a lot of people commented something like, yeah, but I wouldn't do the harmful act. So what I find interesting is you, you know, you're interested in if people are forced to commit a harmful act or not, how do they behave? But a lot of people said like, yeah, but I wouldn't want to do this.
Basically saying like, the, the harm avoidance is so strong that most people felt, or not most, but many people felt, uh, this urge to write that they actually would avoid the harm in the first place.
Jonathan Berman: Right. But it's interesting. Uh, so the trolley dilemma, we compare people either pushing zero, one person off the bridge or pushing one or two people off the bridge, uh.
And the, the people who had the option between one or two people, a lot of them picked two people. You know, they wanna push off two people because they would save that much more people. But that they write, well, I wouldn't do this, you know, which is a little peculiar. Well, if you wouldn't do this, then why are you pushing two people off the [00:31:00] bridge?
Um, it got me interested in thinking about how, if you were given three options, no people, one person, two people with varying, um, live saved in accordance, how would people rank these options? Would they rank? I think a lot of people would rank no people, but then would they rank two people ahead of one or one person ahead of two?
Uh, given the, given the, I should say, um, what I would don't wanna get across is that, um, the. You know, there's a different amount of lives being saved when you push one person over two people in the scenario that is meaningful. Um, and, and so I don't know how people respond to that. That's kind of one, one open question that is, is, is quite important.
Oh, so
Benjamin James Kuper-Smith: you didn't do that in the paper, right?
Jonathan Berman: No, we didn't do that in the paper. Uh, but it's something that it's kind of been in the back of my mind thinking, well, maybe this is something I ought to follow up on. Um.
Benjamin James Kuper-Smith: Mm-hmm. [00:32:00] Uh, do you have any follow ups planned or,
Jonathan Berman: I have vague ideas of, of, of additional questions to study.
Um, that's one of them. How do people rank, uh, these options and how consistent are they in between ranking the options and choosing between two options? Um, and so, um, so I'm not, I'm not quite sure. I don't, that hasn't crystallized in my mind yet. The other. Question I was thinking about, um, uh, uh, earlier today was, um, so I think one question is, when application of this work has to deal with, uh, something like technology, okay, so if we think about a new technology that's introduced, um, let's just say, uh, at some point in time, okay, I'm gonna take, um, uh, uh, take a [00:33:00] very kind of toy example.
Let's just say plastic surgery. Okay. Plastic surgery wasn't a thing or it was being developed. And now let's say they are very good new methods of plastic surgery. Uh, you can imagine some people saying, well, we shouldn't use this at all. You know, people are very reluctant to introduce this new technology because they see kind of, well, if we introduce this, you know, people are going to then change their appearance in ways that, you know, I find objectionable, uh.
And so in the question of should we allow plastic surgery, yes or no? Uh, I think a lot of people would say, would say when it was introduced, may have said no. Okay. But then you say, well, um, well what about people who are severely disfigured? They get into a car accident or they have severe burns. Um, a a and now you say, well, that seems like a reasonable case.
Okay.
Benjamin James Kuper-Smith: Like an exception, almost all
Jonathan Berman: an exceptional. And so you, so you say, we'll allow that. And [00:34:00] now that, that's been allowed, okay, once that's allowed, are they more willing to accept plastic surgery for other cases as well? Okay. And so that's, there's something there that's a little bit synonymous. It's kind of once, uh, once something has been, has been, if you are very anti.
This new technology, once it is used for something, are you much more willing to use it for other things as well?
Benjamin James Kuper-Smith: Uh, so is that the, basically the s slippery slope? Um,
Jonathan Berman: it is a slippery slope type of explanation. I'm not sure if's been framed in, uh, uh, the exact same way as that I've been doing.
Benjamin James Kuper-Smith: But that is the kind of same idea, right?
Jonathan Berman: It's the same idea. It's much more of, yeah. And I think that there is a slippery slope aspect to, to, in some ways I think what we've done can uncover some of the mechanisms behind that slippery [00:35:00] slope thinking. Uh,
Benjamin James Kuper-Smith: I see. Yeah. Yeah. I was also when, earlier when you mentioned, um, environmentalism, uh, as one like context, um, that you were interested in, it also occurred to me that it's kind of interesting because I've never, I feel like when the discussion is, for example, whether people should eat meat or not, or how much.
It's never said, it's never framed in terms of how much damage you have to cause, right. It's always in terms of how much you can save. So what I find interesting is that actually the way you said, it was the first time where I heard the, the framing of not something versus nothing. Uh, but in your case I heard it as something versus more of something else.
So that, that's usually something just occurred to me then that the framing of just how you talk about these things should probably have a huge effect on how it's then perceived. Right?
Jonathan Berman: Right. And you can imagine, um, I think that that's kind of a complicated question because our instinct, uh, as you say, is to, to frame something [00:36:00] in terms of, um, either how much harm can you reduce by choosing one option or the other.
Or you dichotomize it, you say, well, eating meat is environmentally harmful, eating everything else is fine, you know, harmful versus not harmful. Uh, and in some ways. That seems like a very nice framing if you want to, um, enact a change, which could then say, well, I, you know, cause people to reduce the amount of harm that is caused, um, but in, in another way, um, you could imagine that kind of obfuscating the obfuscating the reality.
Um, and so, um, a lot of people are very sensitive to certain environmental behaviors that have very little impact. You know, they don't, they don't consider, well, how much, um, how much is this actually [00:37:00]changing? Uh, how much if we kind of went through and did this behavior, you know, the classic example of late is, uh, uh, the plastic straws.
There's this movement to eliminate plastic straws, which yes, you know, eliminating plastic straws is better for the environment than not, but it is very insensitive to the amount of good and harm being done. Um, maybe there is, uh, if you're a greater sensitivity to the amount of good and harm and being done, maybe we could focus our attention on other things that would actually have a bigger impact, let's say, by, um, uh, uh, uh, flying less or, uh, using more fuel efficient automobiles, you know?
Benjamin James Kuper-Smith: Um, yeah. Yeah. But it seems to me that, um, from what you said, I mean, is that maybe why people are using this kind of framing? Because it seems to be, because people are using this framing in terms of, well, if you. You know, do this, then that's [00:38:00] bad for the environment versus this other alternative is not bad for the environment.
So then you force people basically into, uh, I can't remember what the right terminology is now, but this one way of, um, thinking. Whereas if they were to frame it in the other way, then people would probably act quite differently based on
Jonathan Berman: Right. So the
Benjamin James Kuper-Smith: suggestions, right.
Jonathan Berman: Yeah. Yeah. Yeah. Uh, uh, I mean, very much thinking in absolutist terms is, is typically very good for behavior change.
Any type of behavior, it's much more easy to, uh, to reduce your sugar intake. If you eliminate sugar entirely than you say, well, you know, I'll cut down my sugar in half. Um, and I think it's similarly true for a lot of these moral behaviors. It's much more easy to kind of completely change a behavior, uh, if you're much more absolutist about it.
Uh, and so I, I do think that there is some, um, [00:39:00] there are some advantages to that absolutist mentality, but I also think that, that there are situations where it can be, uh, uh, less than ideal, you know, uh, in thinking about these things, um,
Benjamin James Kuper-Smith: what would be an example?
Jonathan Berman: So, uh, the example I'm thinking about is, and this is not a perfect example, but, um, uh, let, let's just say, uh, ordering, ordering items from Amazon.
Okay. And so the question is, well, is Amazon good or bad for the environment? Well, a lot of people's intuition is that Amazon is very bad for the environment, um, because, uh, they ship things in cardboard boxes. Okay? Uh, and that is unnecessary. If I went to the store, I wouldn't have to buy, I wouldn't have this, this, this waste.
And so the, the mindset is, well, Amazon does harm, going to the store, does [00:40:00] not. Uh, but going to the store does do harm. You often have to drive to the store, which we, you know, leases, uh, carbon emissions, if you're, uh, uh, not using, you know, if you're not. Riding your bike or walking. And a lot of people do do that.
And, um, Amazon can deliver these things more efficiently. Um, there's a lot of, a lot of environmental efficiency gained by centralization, by economies of scale. And so, is Amazon good or bad for the environment? That's, that's unclear. That's unclear, but I think that the kind of this, the tendency to dichotomize these things, you see, um, well, cardboard boxes is harmful.
Um, uh, uh, therefore it's bad. You know, maybe people have other arguments. Uh, uh, but that's more of like a, a, I think a lot of times this black and white thinking gets you only a piece of the puzzle [00:41:00] and they're much more at stake that if you encounter, if you. Kind of map out, well, what are the, what are the variety of sources of harms and benefits across these different options?
Can we do a better job of, of seeing, well, what is actually more, uh, um, appropriate or more, uh, morally, uh, uh, correct in this situation? Uh, um, it's not a perfect kind of analogy, uh, uh, but, um, but, but I think it's a, I think it's a useful one. I think it's a useful one.
Benjamin James Kuper-Smith: Mm-hmm.
Jonathan Berman: Uh,
Benjamin James Kuper-Smith: is, is that one you might use an experiment?
Jonathan Berman: Well, I'm not sure. It's just, that's just more top of my, top of the mind to me. Okay. Um,
it's more top of the mind, uh, uh, to me. And one that I think that people tend to, [00:42:00] uh, uh. See it in more black and white terms than, than is is true of reality.
Benjamin James Kuper-Smith: Mm-hmm. Yeah, I could imagine also in this specific, I mean, not, yeah, as you said, it's just an example. Uh, but in that one, I feel like there's probably also a lot of other concerns about Amazon would start pouring into whatever decision making you have about this one topic,
Jonathan Berman: right?
No, that's right.
Benjamin James Kuper-Smith: There's tax avoidance or whatever. Yeah.
Jonathan Berman: Right, right. Um, that's that's right as well. Yeah. I, I mean, it's a big company. There's a lot going on there. Uh, uh, and a lot of people have multiple complaints. Um,
Benjamin James Kuper-Smith: yeah. Yeah. Um, so one question I had about the paper, and this is something that I've been.
So I'm like in, right now, I'm at the end of my second year, my PhD, and I'm at the stage where I already have some results. And the question is, one thing I'm often asking is like, when is a paper ready to be submitted and when do you kind of need to add [00:43:00] another study? Or like, you know, does it need more or less?
Or if you have several studies, do you package them as one or as two? So, um, you already mentioned one interesting thing, which is you said in the peer review process, it seems like some studies fell out of the thing or something. Um, can you just comment on, so I mean in this, in this paper, you have five main studies, if I see this correctly and a few supplementary ones.
Um, can you just comment on like, did you, yeah. Like did you start off with just one and then see where that goes? Or did you already plan out several in advance or Yeah. How does, how does, did that work?
Jonathan Berman: Yeah. So, um. I, I mean, so the way I work is, um, I like to run a lot of studies, a lot of simple studies, and, uh, I do it for a variety of reasons.
One is I really wanna make sure that the effect is robust. I wanna rule out some alternate [00:44:00]explanations that are top of mind. Um, and then I wanna push the effect as far as I can push it, you know, what are the boundary conditions? Um, how can it go one way or the other? That's not the methods that I use is not gonna be applicable for everyone.
Uh, depends on the project. It depends on your subfield. What are the norms of the field? What do reviewers kind of like to see, um, in this project? Uh,
I have to go back and think, uh. You know, the, the, the other thing about this project is that the course of the paper, you change your theorizing as you go along and you write more studies. We actually initially started out with a narrow, a narrower prediction, okay? Mm-hmm. Um, and we thought that, um, uh, people were [00:45:00] reluctant to commit harm when they could avoid it.
But when they can avoid it, they will commit more harm. So long as it provides an efficiency gain relative to, um, and we do find that, um, but we also find that. And sometimes they do commit more harm even when there isn't an efficiency gain. Uh, and so we initially didn't have those studies in until, uh, Daniela, my co-authors said, you know, I was curious.
I ran this without the efficiency gain. And we still find the effect that people are willing to commit more harm when harming is unavoidable. Uh, and so that caused us, well now we have to design a study to really, to directly test that. Um, and I always had, so, so most of the studies are just the basic effect, and then we have really two studies that go beyond it.
So we have a few studies that [00:46:00] demonstrate the BA basic effect and show robustness. And then we have these two studies that try to go beyond it. One is this efficiency gained versus efficiency of loss. Um, and the last one is the one about protected values. I always knew in the back of my mind as soon as we got started that we would need to examine protected values.
Um, so that was there kind of after we really got started, it was really, you know, the first step is always let's demonstrate this basic effect. Let's make sure the basic effect is there and robust, and then let's try to dig a little deeper. Um, mm-hmm. And so that's really kind of how. How that design went.
Uh, but like I said, you know, different researchers have different methods. They have different ways of going about things. Some researchers are very good at just running one or two studies and then getting that published. Uh, but, uh, I am just much more neurotic than that. [00:47:00] Uh, I really need to, like, I can't, it doesn't sit well with me unless I run more studies.
Uh,
Benjamin James Kuper-Smith: yeah. I mean, yeah. Maybe for, for context, I mean these are the kind of studies where you can do a quick online study that takes people three minutes or whatever, right. To fill in this isn't some electrophysiology and animals away.
Jonathan Berman: Right.
Benjamin James Kuper-Smith: Take like two months to train the animal or something.
Jonathan Berman: Right. And we're not running field studies here.
These are very simple scenario online studies. Um, uh, and the part of the reason why I do this type of research is so that I can run multiple studies as well and then be more confident about what I'm putting out there into the world.
Benjamin James Kuper-Smith: Was that, uh, I have, I've, I haven't read your, like, earlier papers. It was that kind of approach, something you had early on.
Um, 'cause I feel like this kind of don't exactly know. When did you do your PhD?
Jonathan Berman: Uh, I grad from 2009 to 14 was my PhD.
Benjamin James Kuper-Smith: Okay. So online studies were already pretty common or accepted then? I can't, I [00:48:00] dunno exactly what
Jonathan Berman: Well, yeah, yeah. It's interesting 'cause that was really the first paper I published was, uh, there's one study that I ran online that was my first experience with online studies.
And so I was really at the birth of that. And, uh, I mean, uh, for different researchers, I mean, you should try to expand your methods. You should try to use different types of, uh, experimental methods if you're an experimentalist. Um, uh, as much as possible. Uh. And this is more of a triangulation between, you know, the type of research I like to do in terms of methods and also, uh, kind of findings as well.
You know, um, I do a lot of person perception work. How do we think about others? And that lends itself very well to scenario studies two. Um, and so it's really a marriage between the topics and your, your [00:49:00]preferred or what you feel is the best method to study that. Uh, yeah. Yeah. Um,
Benjamin James Kuper-Smith: yeah, I think I should also do more studies like
Jonathan Berman: this.
Well, it,
Benjamin James Kuper-Smith: for me, the data, data now is always the most boring part for me. So
Jonathan Berman: it's,
Benjamin James Kuper-Smith: uh, I spend too much time programming right now.
Jonathan Berman: It's, I mean, it, the difficulty is that it seems easier than it really is. You think, oh, I could just write a scenario and have this, but, but it's taken me many years and even now I make, you know, more mistakes than, uh, than not mistakes, you know, than, than not.
Benjamin James Kuper-Smith: Yeah. I know. That's, that's one thing when I, um, one thing that I find interesting about this say is just how, how simple it is, uh, as a ba and, you know, obviously don't mean that in a bad way. It's, it's just you have these basically five questions, more or less that you ask a bunch of people and then they say yes or no more or less, or option one or option two.
Um, well, I would,
Jonathan Berman: yeah, [00:50:00] I will say as well, go ahead.
Benjamin James Kuper-Smith: Yeah, I was meant to say, I think it's great that you, um, can. Answer a question with something as basic as that. Like, you know, it doesn't, you don't have to have this huge study with super complex statistics and whatever. Sometimes something like this is exactly what you need to address The question,
Jonathan Berman: part of the reason why I was fascinated by kind of the field of judgment decision making to begin with is exactly that reason.
It was, wow, these people are making these really big and convincing claims over these very simple scenario studies. You know, Kahneman and Ky. You present people with a gamble and then you can make these very, very deep claims about human psychology. Um, but I mean, it's also, it's also, um, rhetorically compelling, you know, the simpler yeah, you can be, [00:51:00] the more, um.
Uh, people will latch onto your finding. Um, it's this, it's this kind of a little bit of a tension between the fear from the experimentalist, from the researcher is always like, I don't wanna oversimplify this, because then people won't think it's that, that interesting. Um, but that's, I think, is a much bigger fear than is typically met in reality.
Um, and that if you have something interesting, if you have something that you think is interesting, that you're convinced is interesting, simplify, simplify, simplify. That's always better.
Benjamin James Kuper-Smith: So you didn't get any criticism that this was too simple or, or something like that, or that you in the review process or something like that?
Jonathan Berman: No. If, uh, and in fact, and I'll read you, uh, uh, something from our initial, uh, review.
Benjamin James Kuper-Smith: Mm-hmm.
Jonathan Berman: Um, let me just see if I could find [00:52:00] it. Uh.
Uh, I may not be able to find it, but
Benjamin James Kuper-Smith: mm-hmm.
Jonathan Berman: Um, basically in the initial review we got was this notion, this, this, um, this idea that the editor and the associate editor was like, this is one of these simple distinctions that is meaningful. You know, it's a simple distinction that is meaningful and in some sense, some senses that's,
Benjamin James Kuper-Smith: sorry.
You mean there's distinction between avoidable harm and unavoidable
Jonathan Berman: harm. Right, right.
Benjamin James Kuper-Smith: Yeah. Okay.
Jonathan Berman: Um, uh, in some sense that's what you want. You know, it's a, it's something that is simple, but still meaningful.
Benjamin James Kuper-Smith: Yeah. So what, so one thing that relates to this that I wanted to ask is that, um, so one kind of, how should we say, general.
Uh, it's a conflict almost that I have in my own research is that I'm, I'm really trying, so I come from psychology, um, but I'm reading a lot of [00:53:00] evolution biology or economics or whatever, which is, which, you know, often relies on formal models to make these kind of claims. And one kind of real struggle I have is like to figure out when a formal model is appropriate and when it's, uh, maybe unnecessary or something, or, or not.
The, the research area isn't ready yet for a formal model or something like that. Mm-hmm. And I was just curious how you thought about this because you, you know, have a background from in economics, which is much more, you know, maths heavy, let's say. Um, but then the study is, I mean, well not at all that, right?
So I was just curious how you, how you think about this, this Yeah. When to use it, when not, or whether you considered using it for this paper or,
Jonathan Berman: right. So even though I have some background in economics, I, um. I've, I very much now consider myself a psychologist. Um, and I never really [00:54:00] learned to write formal models.
Um, and, uh, my kind of view on this is that, um, it, so, so some people they think better in formal models. You write a model, that's how they understand what's going on. Um, for some people it, it's, it's, you know, gibberish to them. Um, um, so one, it depends on kind of the audience you wanna reach. Um, and if you can do both, that's great.
I don't see any harm in that, uh, particularly if you could do that well. Um, and, uh, uh, uh, what you think better expresses your idea, um, your theorizing, um, the, I. Certainly in some of my work, a formal model could be written, uh, but I just don't think I needed that for, for, for the papers. [00:55:00] Um, so that's, you know, much more, I would say one of a question of what you believe better expresses your theorizing or idea and, uh, to who your audience is.
You know, it's a combination of, of both of those. Um, what audience do you wanna reach as well? Um, and there is, uh, there is a good amount of psychologists who rely on formal models as well, you know, um, uh, so I don't have strong opinions as is, uh, as my way of. Uh, of getting to that. Yeah.
Benjamin James Kuper-Smith: Yeah. I think part of, uh, you answered part of a false assumption I had, so I get, I think also because you're at a business school.
Jonathan Berman: Mm-hmm.
Benjamin James Kuper-Smith: Um, and I, so I don't really know what a marketing department exactly is or what marketing, because you know, it always sounds like advertising, but I'm sure it's not that. Um, so my assumption was also still that you, um, would consider yourself more an [00:56:00] economist or something like that, and that would be more your primary audience.
So one question I actually had was also like, why some, it's to psychology journal, but I guess you answered it, you consider yourself, right? A psychologist?
Jonathan Berman: Yeah. So, uh, marketing is this weird department as well, um, that there are different, there are different kind of camps in marketing and it's very specialized.
Um, um, there are people who are much, much more economics heavy and basically do eco applied economics. Um. Or statistics heavy, and they do applied statistics. Um, whereas the, the, the camp that I associate myself is much more psychology, uh, or, or psychology. And even in some sense, sometimes sociology, sometimes anthropology, um, uh, driven as well.
Um, yeah. Yeah. The first, one of the first things you teach students is they enter the marketing classes. The marketing is not [00:57:00] advertising. Um, uh,
Benjamin James Kuper-Smith: but why, what is it then? So why or why is it, why is it confused with it?
Jonathan Berman: Well, marketing we consider to be, um, a bar. You know, a simple way to define it is the creation and communication of, of, of value, of consumer value.
Okay?
Benjamin James Kuper-Smith: Mm-hmm.
Jonathan Berman: Uh. So, uh, marketing encompasses advertising is one of that, that's the communication of value. Uh, but marketing is also kind of changing your product to meet the needs of consumers. It's, it's as one of, uh, one of my colleagues, uh, ODed, Conans Berg says, it's the interface between the company and the consumer.
What is the consumer? So you're trying to figure out what does the consumer want, what do they need, and how can we change our product offerings to meet the needs of those consumers, um, in order to create long-term. Uh, long-term value. [00:58:00] Um, so yeah, it's what, what is, what, what is the product? Who are we going to target?
What do these target customers want in a product? Uh, what price are you gonna charge? That's a marketing question as well. Um, not just in terms of, um, you know, what price can we charge? Um, where do we distribute this product? Uh, we consider this, we consider these questions all to be marketing questions.
Um, whereas something like operations is much more about, um, once you've decided or how can you, how do you go about distributing your product, right? Um, yes, it's tied to where you distribute it to make it more cost effective, but, but marketing is interested in, well, we need to put the product where the consumer shops.
Uh, we need to figure out that question. Um,
Benjamin James Kuper-Smith: mm-hmm. So then how does a paper on moral decision making, [00:59:00]because this is then very, you know what most of what you described is fairly applied science. Right?
Jonathan Berman: Right.
Benjamin James Kuper-Smith: Whereas what you're doing here is basic, uh, moral cognition or however you might want to call it.
Jonathan Berman: Right. So, um, so the way I, I, I, the way I kind of associate this with marketing is that, um, one, the easy way is to say that I study consumer ethics. Okay? Um, and so it's a question of to what extent, uh, are consumers willing to engage in ethical behavior? Um, in what situations will they engage in ethical behavior?
Uh, how much can we anticipate consumers to engage in this behavior, uh, an ethical or moral behavior? And, um. Uh, and so this related to this product project. Uh, some people kind of define consumption. You know, any consumption activity you [01:00:00] engage with is, um, uh, you could argue has some moral implications to it.
Um, uh, or at least, you know, uh, some definitions say that consumption is inherently tied to environmental harm. Um, um, at least in the vast majority of cases. And so, uh, so that's the question that I am, I'm interested in. Um, that's my question. I'm also interested in, you know, questions about kind of the market dynamics in, in general.
To what extent can we, um, to what extent, extent. Can we rely on market dimen dimensions to create a moral and just world? What are the limits of that? Uh, market interactions, I should say. Um, uh, I would say this paper speaks to that a little bit. Uh, you know, in the [01:01:00] sense that people kind of, this more, this, this kind of, when do people think doing more harm is a good thing when it provides this efficiency gain to doing less harm?
Benjamin James Kuper-Smith: Yeah.
Jonathan Berman: Uh, and I think that that relates to our economy very broadly, um, and how people see that. Yeah.
Benjamin James Kuper-Smith: Yeah. But do you plan on also doing more applied studies in the future about this topic also, or, um,
Jonathan Berman: we'll see, you never, you know, research is so unpredictable that first, the, the, the right idea has to come to you, uh, and then you try it and you see if it works.
So I would like to continue. More applied domains related to this work. Uh, but who knows if and when I'll actually get there.
Benjamin James Kuper-Smith: Yeah. Yeah. So it's not, it's not like this is part of a, you know, sometimes you have like one big idea and like, okay, this is like I'm testing part one of this big idea. [01:02:00]
Jonathan Berman: Right. Yeah.
That's never been my forte. Uh,
Benjamin James Kuper-Smith: okay.
Jonathan Berman: Uh, uh, in some sense, you know, I like exploring too much. Uh,
Benjamin James Kuper-Smith: I think I know what you mean.
Jonathan Berman: Yeah. So I probably could have published more papers if I, if I took that route. But, um, uh, yeah. No, I like exploring too much. Yeah.
Benjamin James Kuper-Smith: Uh, maybe one thing I was, uh, I was also wondering about is that, uh, it seems like you're also, you know, fairly at the forefront of the open science stuff with, in this case, your, uh, data was open for all of the experiments. Uh, your materials were all there, there were pre-registered and there were also, as far as I can tell, well pre-registered not the kind of preregistration that sometimes pre-registration aren't quite as precise as maybe they [01:03:00] should be.
Um, is that something, uh, so like, again, I, I assuming that you came from more kind of economics kind of background or something, I feel like this is maybe something that psychology do is doing what, probably more than most fields. Um,
Jonathan Berman: yeah. So yeah. I was, I, I would say I was at the forefront of it, but now it's becoming a lot more standard now I am one of many.
Yeah. Um, so I was lucky enough when I was doing my PhD, um, I had a lot of exposure to Yuri Simonson, who is one of
Benjamin James Kuper-Smith: Right.
Jonathan Berman: The leaders of this movement. Um,
Benjamin James Kuper-Smith: so he wrote that paper on the
Jonathan Berman: p hacking. Yeah. False positive Psychology. He's written a, a number of papers, uh, him and life Nelson, but
Benjamin James Kuper-Smith: one of the big ones in the field.
Yeah.
Jonathan Berman: Yeah. Him life Nelson and Joe Simmons. I would argue that they were really the ones that, um, spearheaded [01:04:00] this revolution. Okay.
Benjamin James Kuper-Smith: That was in relation to the Bem article, right?
Jonathan Berman: It wasn't, or
Benjamin James Kuper-Smith: was it just afterwards?
Jonathan Berman: It was around the same time. I don't think they had that article in mind when they wrote that paper.
And they did have follow up stuff related to the BE article. Um. But, but it was almost my, my memory of it, it was concurrent. Um, uh, and Yuri Simonson had been doing some, uh, debunking work a few years before then. Um, and so. So I was very lucky to, to be exposed to that very early on and really, um, uh, try to, to, to follow that as much as I could.
Um, and over the years I've just become more and more, uh, passionate about, uh, uh, engaging in open science, engaging in pre-registration. I am, I've, I've drunken the Kool-Aid, so to speak. Um, [01:05:00] and, and, and very much, I very much think that it's a good thing and that psychology is, is leading the revolution, you know?
Um, yes. Uh, economics had advantages on psychology. They're, they were much, much more transparent with, uh, their data that psychology was. Uh, but uh, I think that they also have a lot of issues that they're not working through as much as psychology is. Um. That's my, that's my, uh, impression. Uh, I don't know the details there.
So, so, yeah. Yeah, yeah. That was a big, I now have a few published papers that have preregistration. Um, with it. I try, uh, I, I upload all my data, uh, to reviewers. I think that's very important that reviewers have access to data. Um, uh, I wanna show, put all my cards on the table, you know? Um, I don't see why not.
Um, I think [01:06:00] people think that, um, that they're at a, you know, that they're at a diff disadvantage that publishing is so hard to begin with, that they want kind of, they're afraid that if they show a flaw somewhere, uh, that reviewers are gonna pick up on it. I mean, if that's the case, then you should be, you know, one, I kind of trust Rev.
This is another reason why I run many studies. I wanna demonstrate this basic effect is real. It's robust, you know, and if there is a study here or there that doesn't work perfectly, then uh, I'm fine with that. And wherever it gets published, so be it. Um, uh, but now, you know, I just, we just had kind of, one of my flagship conferences that I attend is the Society of Judgment Decision Making, uh, conference.
And in that conference, uh, almost everything is pre-registered. That's now the norm. It's an exception if you come [01:07:00] across a talk that doesn't have pre-registration involved with it. Uh, and so it's just becoming more and more the norm. And, uh, and, and I'm very happy about that. I'm very happy about that.
Benjamin James Kuper-Smith: Is that conference organized by the people who do the judgment decision making journal?
Jonathan Berman: Uh, well, it, yeah, it is tied. So the judgment decision making journal is, is really done by John Baron, although he's kind of passing the torch, um, uh, in the process. He's kind of, uh, uh, uh, passing that on. Um, and he is one of, uh, the leaders of the judge and decision making conference. He isn't, um, I don't think he.
I don't think he has that much day-to-day, uh, involvement with the conference or with the organization, but, um, he is, he is one of [01:08:00] the, the big names tied to,
Benjamin James Kuper-Smith: yeah, I mean, the reason I was asking is just because that journal is also one of the early ones that had registered reports.
Jonathan Berman: Yeah.
Benjamin James Kuper-Smith: And they, I mean,
Jonathan Berman: they're very open with their data and
Benjamin James Kuper-Smith: their website is the best website ever because it looks like it's designed in 2003.
But, um, they, they're really just, despite the website looking old, they're really on the forefront it seems of all this stuff. I think you have to have open data and all that kind of stuff to publish there. Something. Um, yeah, I was just curious whether maybe that had an effect on the conference being like that also.
Jonathan Berman: I think it's just the, the people there are, um, just passionate about. Uh, I, I, I think it's a, it's, it's a type of field and a type of conference where, um. Uh, uh, and, um, I think actually the open science revolution has actually kind of started with a lot of those members, but has made the community a little bit more cohesive, you know, rather [01:09:00] than feeling petty that to this other person is publishing more than you, you go to the conference and people really feel, um, uh, proud of the work that's being done.
Um, and uh, particularly they like to see good research, you know, um, and, uh, uh, uh, and there's not that much pettiness involved among the researchers just so long as they see something that they like and they appreciate, you know? Uh, so, so I think the ideals are, are, uh, people's heart are in the right place.
Um, and to the extent that I can be a associated with that and that feeling, that's something that I would, I would love, I would love to be associated with.
Benjamin James Kuper-Smith: I'm curious about, so you mentioned, so one slight, um, one slight problem I see [01:10:00] is that, so you, you said you like to do lots of different small experiments, right?
So I mean, do you then report all of them or what do you do with some that maybe didn't make it in the paper, like what happens with
Jonathan Berman: Right. So, um, uh, uh, people have different feelings about this. Uh, I have no problem, uh, with people having a sizable file drawer. Okay. This is kind of the typical file drawer problem.
Uh, there are many reasons why I run studies that don't get published. Uh, so one reason is you just didn't design a great study, you know? Um, and a lot of times you design a study and you realize, oh, that wording is very confusing that I've used, or those dvs aren't really what I'm trying to measure.
There are all sorts of reasons why studies end up in file chores. Or studies don't work or do work, you know, um, that could end up in the file drawer, uh, for [01:11:00] any number of reasons. Um, uh, I probably do put my best studies forward. Uh, but, uh, I'd like to think that, uh, they all would, or, uh, just about every one would, and I wouldn't know which one wouldn't replicate, you know?
Um, if I feel uneasy about a particular study not replicating, then uh, you know, that's a sign that maybe I should run it again, or I should, um, think differently because now you know that. There are these registered reports out there that if you're doing something that you think is impactful, people will try to replicate what you're doing.
So, you know, and my studies are very easy to replicate, um, and, you know, takes, uh, uh, not that much money, not that much time to program it. And so if I'm publishing a study, I wanna feel pretty confident that if [01:12:00] someone replicates it, it's gonna, it's gonna work. Um,
Benjamin James Kuper-Smith: but where do you draw the boundary? Maybe?
So let's say you have a study where you, you know, it's just, I'm wondering like, where do you draw the boundary when you say maybe this study wasn't designed properly? Do you decide that then before you look at the results or after you look at the results?
Jonathan Berman: Oh,
Benjamin James Kuper-Smith: because it has, like, say the results are kind of like a bit messy and then you say, well maybe it's because of this wording, or like, how do you navigate this kind of field of.
Jonathan Berman: Yeah, I mean, there are always gonna be these judgment calls. Um, you know, I often think that there should be like a, um, someone should have a column that's kind of like data ethics, where you write in these kind of questions, like, this is what I did, what should I do? You know? Um, uh, so I I I, I tend to think that there are gonna be these judgment calls.
I think that, you know, the way I think about it is that a lot of times you, it's clear after the fact what was, um, [01:13:00] confusing or what could have been confusing a lot of times. There's another confound that you didn't really think about and you think, well, maybe people are thinking about it this way. Um, you know, these are difficult questions.
Um, there's no easy answer to them, but what I don't want, um, you can hear my little baby. Yes. Um, what, what I don't want is for someone to say, well, if you change this wording, your effect goes away. And this wording is a better wording. You know, I don't want the effect to be due to a misunderstanding. Um, and it's another reason to post your materials, to post your scenarios.
Um, so people can see, oh, uh, there doesn't seem to be a confusion here. Or, you know, if you, uh, the, the one rea I like preregistering, I like posting because it causes you to think more deeply about what you're doing. Um, [01:14:00] uh, and I think that that's a good thing.
Benjamin James Kuper-Smith: Yeah. Def But do you, so some of those studies that you haven't published, are those also pre-registered or
Jonathan Berman: Some are, I mean, I have plenty of, um, yeah, yeah.
Lucky enough, in this project, almost every study we ran kind of worked. You know, there may have been some that, that didn't quite work, or a lot of times really you get like a directional effect and it's just not significant. So you think, well, you know, I need to increase my sample size and run this again, or maybe I need to change the wording to make it more clear what's going on.
Um, uh, so yeah, there's some studies, some studies. Did it work? Uh, and there, there, there, you know, I think typically what we try to do is you see like there's these supplemental studies that, um. I wish that people were much more accepting of, of null [01:15:00] results. Um, a lot of times you run a study, you get the results and something's not quite right, and you're kind of pretty embarrassed that you didn't see the confound beforehand or you didn't see that way things could be, um, differently.
So yeah, it's, it's this, these are, these are complicated questions. There often aren't easy answer here. Um, uh, but I think people, you know, if you really are. The problem I typically have is more under confidence in my effects and overconfidence. Um mm-hmm. Uh, but that's gonna vary by person. Um, like if I had my way, I would never have submitted any paper I've written because there's always something nagging at me that could be, you know, well, isn't this just due to this?
You know, and then you write something, oh, it's not due to that. Well, isn't this just due to this little, you know? Um, I [01:16:00] think I've now gotten over that, but that was a much bigger problem earlier in my career. Um,
Benjamin James Kuper-Smith: yeah. Yeah. I just find it. Yeah. I, I mean, like, the reason I'm asking this is just because I, you know, like, I like the whole, I think pre-registration is, um, a great idea when used correctly.
Um, and, but yeah, it's just sometimes a bit difficult to navigate. Exactly. Um, yeah. When do you pre-register something and Yeah. Do you publish everything or, because I think for example. Is it an os? I think so far I've used the, you know,
Jonathan Berman: IOSF? The
Benjamin James Kuper-Smith: OSF. Yeah. With them. I can't remember like one of the, they're the main one, right?
Where
Jonathan Berman: Yeah.
Benjamin James Kuper-Smith: Yeah. I think I've used their standard template, which is also longer than, for example, what you used with as predicted. Um, and I think there, they say something that this, this will be pub or I can't remember, but like if you, or was that just for a registered report? Whether you have to say why you didn't publish it.
Jonathan Berman: [01:17:00] Uhhuh,
Benjamin James Kuper-Smith: I can't remember. Uhhuh, I'm a bit confused about the whole thing.
Jonathan Berman: There are, there are different camps and, uh, with as predicted, no, no, you don't have to publish it. Um, and there are good reasons why you wouldn't publish something. Um, yeah, and I, that's where I stand on that decision. I think it's, having a file drawer is perfectly acceptable.
Um, uh, because research is a process. Research is a process. Uh, and I think for them, the preregistration is a lot more about, um, collecting the number of people you say you're gonna collect, examining the dvs you say you're going to examine and not about having the full theory ahead of time, you know about what's gonna happen.
You know, that they would be, in many cases fine with saying, I don't know if this is gonna come out this direction or this direction, but this is the sample [01:18:00] we're collecting. These are the dvs we're analyzing. This is the Yeah,
Benjamin James Kuper-Smith: I've interest, I've never used as predict before and, well, I still haven't, but I, um, when I read your preregistration it, um.
I did find it interesting that, for example, for sample size Yeah. They, they decided to state it and then in brackets you don't have to justify it or something like that. Whereas with the OSF, you have to justify basically why you. Right.
Jonathan Berman: And there's another, there's a, there's another reason for that, which is it's hard to justify sample sizes, you know?
Yeah.
Benjamin James Kuper-Smith: It's, yeah. Often impossible, basically.
Jonathan Berman: Yeah. I don't know how big this effect is, so I can take a guess, you know, and a lot of times, uh, uh, the way I'll run 400 people on this study, where am I picking that number? I don't know, you know, but, um, it's better than 300. It's not as good as 500. Um,
Benjamin James Kuper-Smith: yeah. Yeah.
Jonathan Berman: Five hundred's better than 400, but not as good as 600. So,
Benjamin James Kuper-Smith: yeah. [01:19:00] Yeah. And I think, yeah, I mean, yeah, sample size, you know, then you say something like, well we assume a minimal small effect size of cos D equals 0.3 or something. And that's also just basically arbitrary. I, uh, uh,
Jonathan Berman: uh, before you study something extensively, uh uh, I have been wrong about the effect sizes a hundred percent of the time.
Benjamin James Kuper-Smith: Yeah.