38. Keno Juechems: Where does value (in RL) come from, optimality with finite computational resources, and learning as a PhD student

Keno Juechems is a Junior Research Fellow at St John's College in Oxford. He studies how humans make decisions, using computational modelling, behavioural tasks, and fMRI. In this conversation, we talk about his papers  "Optimal utility and probability functions for agents with finite computational precision" and  "Where does value come from?", and various related topics.

BJKS Podcast is a podcast about neuroscience, psychology, and anything vaguely related, hosted by Benjamin James Kuper-Smith. New episodes every Friday. You can find the podcast on all podcasting platforms (e.g., Spotify, Apple/Google Podcasts, etc.).

Timestamps
0:00:05: Where does the name "Keno" come from?
0:01:47: How Keno got into his current research area
0:14:09: Start discussing Keno's paper "Optimal utility and probability functions for agents with finite computational precision"
0:26:46: Rationality and optimality
0:38:58: Losses, gains, and how much does a paper need to include?
0:51:04: Start discussing Keno's paper "Where does value come from?"
1:10:28: How does a PhD student learn all this stuff?
1:19:56: Resources for learning behavioural economics and reinforcement learning
1:25:42: What's next for Keno Juechems?

Podcast links

Keno's links

Ben's links


References
Juechems, K., & Summerfield, C. (2019). Where does value come from?. Trends in cognitive sciences.
Juechems, K., Balaguer, J., Spitzer, B., & Summerfield, C. (2021). Optimal utility and probability functions for agents with finite computational precision. Proceedings of the National Academy of Sciences.
Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica.
Keramati, M., & Gutkin, B. (2014). Homeostatic reinforcement learning for integrating reward collection and physiological stability. Elife.
Lewis, M. (2016). The undoing project: A friendship that changed the world. Penguin UK.
Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction. MIT press.
Thaler, R. H. (2015). Misbehaving: The making of behavioral economics.
Trepte, S., Reinecke, L., & Juechems, K. (2012). The social side of gaming: How playing online computer games creates online and offline social support. Computers in Human behavior.

https://en.wikipedia.org/wiki/Indifference_curve
David Silver's reinforcement learning course on YouTube: https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ
Chris Summerfield's course How to Build a Brain: https://humaninformationprocessing.com/teaching/

  • [This is an automated transcript with many errors]

    Benjamin James Kuper-Smith: [00:00:00] But I, I think I have to start with something, uh, very boring. But why exactly is your first name? Keno. As far as I understand, you're German and I've lived in Germany for 17 years or so. I think you're the first person I've ever actually heard the name. 

    Keno Juechems: Yeah. So it's, it's a bit of an unusual name. That's, that's true.

    So, um, I'm, or 

    Benjamin James Kuper-Smith: is it keno or? 

    Keno Juechems: Yeah, it's Keno. Yeah, exactly. Okay. And, um, I'm originally from the northwest of Germany, so atrophy stand and, 

    Benjamin James Kuper-Smith: okay. 

    Keno Juechems: It is a relatively common name there, so it's, it's not super common there either, but it's like a traditional name from that area. 

    Benjamin James Kuper-Smith: I did. Okay. I didn't realize it was actually a German name.

    I thought it was something like, oh my, I dunno. Parents like Japan or something like that. I don't, 

    Keno Juechems: yeah. Yeah. A lot of, a lot of people say that, that it sounds like a Japanese name. 

    Benjamin James Kuper-Smith: Yeah, that, that was my assumption. Maybe. 'cause it sounds like Kendall or something. Oh, okay. [00:01:00] So there's literally no story. It is just from the area.

    Huh? That's weird because my family's also, I mean, not like Northwestern Germany, but. My grandparents' generation are from Sland. Oh, 

    Keno Juechems: okay. Yeah, that's 

    Benjamin James Kuper-Smith: pretty close. Um, okay. I mean, I have no real relation to that area. Yeah. Um, I've been there I think twice or something, but it still surprises me though that I've, yeah, I've never heard the name anyway.

    I guess that's not, 

    Keno Juechems: no, but it, it is a fairly sort of confined area where you would encounter that name. Um, so even in the m stunt, it's probably not something people would generally hear pretty much. 

    Benjamin James Kuper-Smith: Okay. Yeah, I mean also the names that I'm familiar from, from that area are, you know, two generations in the past, so everyone's put basically.

    Keno Juechems: Okay. 

    Benjamin James Kuper-Smith: Anyway, so yeah, as I said, um, I'd like to talk about these two articles, your I, Fred, where does value come from, and optimal utility and probability functions, et cetera. Um, and I thought maybe we can do that by talking about kind of how you got [00:02:00] into that and how those projects started. 

    Keno Juechems: Yep. 

    Benjamin James Kuper-Smith: Um, and maybe the first question is just.

    So in 2012, you have a paper called The Social Side of Gaming, how playing Online Computer games creates online and offline social support. How exactly do you go from there to what you're doing right now and what you've been doing in your PhD? 

    Keno Juechems: Yeah, it's an excellent question. Um, so when I was doing my, my undergrad in homework, um, I worked as a research assistant in media psychology.

    So at the time I was, I was interested in like, yeah, media in general. And I also, before I started studying psychology, I thought maybe I want to do, uh, a politics degree. So this seemed like a good compromise to still be in touch with that. And while I really enjoyed working in media psychology, um, and where I also worked on this paper that we wrote together, I sort of realized that I'm more interested in.

    Sort of neuroscience. Um, so the brain, broadly speaking, uh, was what in interested me more in my studies than, than [00:03:00] medium psychology per se. And then when I did my master's, I did a project, um, with Chris Summerfield, which developed into, into a paper later on. So, um, that's sort of when I got back into the, um, decision making field.

    And I think that was my first sort of ex like, firsthand experience with, um, with that kind of area. But funnily enough, so this optimal utility functions paper builds a lot on, um, what's called prospect theory. So like an economic theory from, um, from the 1970s that sort of, uh, incorporated a bit of, uh, psychology into mainstream economics.

    And uh, that was a paper I wrote an essay on in my very first semester in handbook. Um, and I was really interested, interested in that. So it's kind of funny, that's like full circle. 

    Benjamin James Kuper-Smith: So it seems like you changed direction completely, but you, you know, wandered about a bit and came back to what you started with.

    Yeah. Okay. But what was your, um, I was also just generally curious, um, uh, [00:04:00] on, I think it's the St. John's College or what's the college? Yes. Yeah. Um, it says, my background is in psychology and cognitive science, which sounds very mysterious. Was your master's then in Oxford or was it somewhere else? 

    Keno Juechems: Uh, no, I did my master's in, uh, ble.

    So 

    Benjamin James Kuper-Smith: I should have, I had, sorry to interrupt, but I had a, I I thought, should I do the risky gamble or not of suggesting that you either did your, a master's in cognitive science in marketer book or I think those are the two places in Germany, right? Yeah. Where a lot of people seem to do very well afterwards, after doing that master's and I thought, nah, that's probably not gonna be true.

    Keno Juechems: Yeah, it is. It is. Here we're, yeah. Um, yeah, so like after, after my undergrad, I. Uh, I took a year off and sort of thought about what I wanted to do and I, um, uh, I really wanted to go into research and the psychology degrees, um, as you probably know, are fairly heavily focused on clinical psychology or organizational psychology.[00:05:00] 

    So I really knew I didn't want to continue with that. And then I just looked for good research focused degrees and cognitive science was one of them I'd actually looked at for my undergrad as well. And then, um, I was, I really liked that it's, it's a very, very flexible degree. Um, so you can more or less focus on whatever you like as long as you tick a few boxes.

    Um, and that's where I learned programming and it fit into the program and it was, uh, it was really good to, good to do it there. 

    Benjamin James Kuper-Smith: Can you talk just briefly about the program? Just because, yeah. As I said, like I've. I feel like half the, the people who studied for their masters in Germany, they did one of those two programs.

    If maybe that's wrong, but that's at least the impression I'm getting. So it seems to be a very good program that people seem to like quite a lot. And I'm also curious then Chris Summerfield ist an Ner boy. Right. And I don't think he ever has been. Yeah. So how did that exactly come about then? 

    Keno Juechems: Yeah, so about the program.

    Um, so I think it's. It's an excellent program for, um, if you want to go into research. [00:06:00] So if, if you want to do something else afterwards, like actually work as a psychologist, for example, um, then it, it wouldn't be a good degree. But it, I think it's a good preparation for, for research because it's very flexible and it also, um, at least in the undergrad, really focuses on philosophy of science, uh, philosophy of mind.

    Uh, you have to do lots of programming and mathematics courses and you don't have to do those in the Master's, but you can, and they're sort of easily available to you. And I did did some of those. So it's, uh, yeah, it's really good preparation if, if you sort of want to pursue a career in something that's researchy, even if that's working for a company afterwards.

    And yeah. To, to answer your second question, when I. When I started in oab, uh, some people who were doing their PhD in Oxford, but had done their undergrad in Oab, they contacted the Farha, so like the, the student body of the course and asked whether people would like to do a visit to Oxford. [00:07:00] And I was like, 

    Benjamin James Kuper-Smith: everyone said no.

    Keno Juechems: Yeah, exactly. 

    Benjamin James Kuper-Smith: Why would we want to do that? 

    Keno Juechems: Yeah. So I was one of the lucky ones who, who actually got to visit and I, at the time was thinking about doing their master's thesis, like abroad. Anyway, so I was already thinking about somewhere in the uk and then I just contacted a few people and, uh, Chris was one of them.

    And, uh, yeah, he was fairly open to me doing that. So a year afterwards, I, I came over to do that and, uh, haven't left Oxford since, essentially 

    Benjamin James Kuper-Smith: Chris hasn't left you. It's funny to me how like a lot of these things are just. Similar like all of my master's programs. Like, uh, I had, I had like something where I did two research projects.

    Mm-hmm. And all of them were just, I contacted people I thought, whose work was interesting and then for some reason they responded, which is always, I dunno, I guess like when you are at, at that stage, I didn't necessarily assume they would, but 

    Keno Juechems: yeah, it's, it's easier than, than you think really when [00:08:00] you, you write and it also seems a lot more intimidating to contact these people when you're just doing your masters.

    You know, you dunno how that, that kind of interaction works. So I definitely, 

    Benjamin James Kuper-Smith: yeah, 

    Keno Juechems: it, it took me many hours probably to write these emails to people, 

    Benjamin James Kuper-Smith: which is a, I mean also why that were successful, right? I mean, 

    Keno Juechems: maybe if you 

    Benjamin James Kuper-Smith: just write like you are, Chris, can I come hang out with you 

    Keno Juechems: then? Well, actually I didn't, I, I actually didn't address it to Chris because at the time, um, his email address for the lab was oodle.

    Uh, and it, it's still an inbox that, uh, some people use for the lab. Uh, so I just addressed it as Dear Neuro Noodle rather than Chris. 

    Benjamin James Kuper-Smith: That's, uh, how long did you think about whether you're gonna write that as the, probably 

    Keno Juechems: for a 

    Benjamin James Kuper-Smith: very long, yeah. So can I rewrite this? Well, it seems to work, although we should probably say if there are people applying for master's projects only do that.

    If that is the email address of the lab, just [00:09:00] randomly contact people with G Neuro, I 

    Keno Juechems: probably made sure that it definitely was. Yeah. 

    Benjamin James Kuper-Smith: Okay, cool. So it was it then, I'm assuming, I don't know how, when you contacted the neuro noodle, uh, was it a fairly generic kind of like, Hey, I think your research, your stuff is cool.

    Can I do my master's project with you, or did you come kind of with ideas already to do something specific? 

    Keno Juechems: Honestly, I don't really remember, but I probably didn't come with a very specific idea. Maybe I wrote about some things that I was interested in, um, that I was interested in decision making, and that's, that's what he worked on or still does.

    But I probably didn't have a specific project idea in mind. Uh, but I, I do remember that he gave me a lot of reading afterwards, so for suggestions and asked me to have a look at that, and then we could figure out a project once I moved to Oxford. 

    Benjamin James Kuper-Smith: Okay. So how'd you go from Master's project to PhD, the standard route of applying for stuff or [00:10:00] did he have funding, or how does that work?

    Keno Juechems: Uh, yeah, so it's a bit different from Germany. Maybe the way it works in Oxford because it's an actual program. So in Oxford you, you can apply for open positions, right? And then it's like a, it's like a job. You are basically employed on the project of your supervisor. That's not really something you, you can do in Oxford.

    Um, so you do have to go the official route and, uh, apply through the university program. Um, so I, I did that about a year after I did my project with Chris. I think after I started maybe, um, 'cause it's, it's one round a year and, um. Then I had an interview, and then there's usually a very, like a sort of uncertain long period of time when you get your offer, but you don't have funding yet.

    Um, so Chris said you might be able to fund me anyway, but there's, there's several rounds of internal funding that you might be able to get in the end. So after maybe three months. 

    Benjamin James Kuper-Smith: Oh, sorry. So you got [00:11:00] accepted into the program and then the question was they, like, they said like, okay, in principle you can come.

    So the, the, the program isn't directly, acceptances of the program isn't directly linked to actual funding. 

    Keno Juechems: No, unfortunately not. So I think they accept about. People every year and maybe 10 would get full funding from various sources. And I dunno how the other 10 really drew it, because you also have to pay tuition fees in Oxford, which is, uh, I think about 6,000 pounds, which is just a home fee.

    So if you're, if you have citizenship that isn't European, then it's at least twice as much, maybe even more. Yeah. So it's quite expensive for, for a program where you think, I don't actually do any, I I don't actually take any teaching. I'm just working here, so it seems like quite a lot of money if you don't get funding and uh, which it definitely is a lot of money.

    Yeah. But I was, I was lucky that after a few rounds, I, I did get funding for three years. And that, that's another thing actually, that the [00:12:00] people who do get full funding, like me, they would have three years of funding, but most people need about four years. And I also needed four years. Um, so for, for the next, uh, for the last one year, you kind of have to either fund it yourself or get funding from your supervisor.

    So for me, it was a bit of a mix in the end. And I was also able to extend my funding a little bit because I, I did a, a visit to another lab during my PhD, uh, and then they sort of extended it afterwards. But that, that definitely is a, is a bit of a problem. 

    Benjamin James Kuper-Smith: Yeah, that's really weird, like in a way, like it's, especially in the UK I guess it's in Germany it's becoming more common also, but it, you know, these three year PhD programs, I mean, you can finish in three years, but.

    It seems to me for most people I've heard also who've done very well in the PhD, right? Like for example, you know, you had some very good publications out of it. It's not as if you were just sitting around all day, didn't [00:13:00] get anything done. Most people say it's, yeah, you need like four years basically. And 

    Keno Juechems: yeah, 

    Benjamin James Kuper-Smith: so I wonder sometimes why they do these three year things when basically everyone kind of needs more anyway, I guess.

    So they can offer more positions, but 

    Keno Juechems: yeah, I'm not really sure. And it, it also really depends on what kind of PhD you're doing. So in my program it's, um, it's for the entire psychology department. So if you're doing something like me where you just, you can do your experiments and some people might just do behavioral experiments that, um, aren't.

    All that time intensive to run or don't take as long to analyze. So you can definitely, if you really, really want to, you can probably do that in three years. But then others, um, are doing, um, are working with, uh, clinical populations. So, um, that will take at least a year or more just to set up your study and then you have to collect the data and find patients who want to enroll in your study.

    And that's almost impossible to do in maybe [00:14:00] even four years. 

    Benjamin James Kuper-Smith: Yeah. 

    Keno Juechems: Some people would even need more time and some people do get an extension because of those circumstances. 

    Benjamin James Kuper-Smith: Yeah. Anyway, so how, so, so before we start recording, you said that the optimal utility paper kind of came first in a sense, or that those, you were early on more interested in those kind of topics.

    Keno Juechems: Yeah. So in a way you can kind of, so like, sort of biographically, it fits very well with the very first study that I did with Chris, um, which was. Uh, another gambling study where we were looking at how people change their risk preferences based on how much money they've accumulated over repeated gambles and where, where, and how that might be represented in the prefrontal cortex in the brain.

    And so I spent a lot of time working on, uh, utility functions and sort of economic decision making during that time. Then there was a bit of a break that's led to, uh, eventually to that. Where does value come from? Uh, paper. And [00:15:00] then, um, this optimal utility function sort of circled back a little bit, but started off as a project that, um, that someone else was leading actually.

    And I only joined it sort of halfway through. So this was like a very, very collaborative project that I was sort of later, um, sort of tagged onto because I, I had already worked on something similar. 

    Benjamin James Kuper-Smith: I see. So, I mean, I guess maybe if you weren't. I mean, you might not have been then there right from the beginning, but what was kind of the, the goal of this project or.

    Yeah. Do you, do you know how it started or, I mean, I, I, I guess I'm kind of asking because I'm off right now, you know, I'm also my PhD and I'm, you do these research projects and sometimes you have things that are very kind of clear cut and you have this idea you do it and Hmm. And sometimes you have these things that kind of shift and change as you do them and, you know, you don't even know how many papers it's gonna be or Yeah.

    You know, all these kind of things. So I'm just curious kind of what it was like for this [00:16:00] one. 

    Keno Juechems: Yeah, so for this one, um, it sort of started off as part of the research agenda by, um, Ben Hoso, who's the, one of the senior authors on the paper who was doing a postdoc with Chris. Chris at the time. And he is been working a lot on how noisy encoding of decision variables.

    Um, so for example, we may not, like, we might see a number of dots on the screen, but. There's a bit of uncertainty about how many we might see at any given point in time just because of some variability in, in our internal processing. So he is worked a lot on, on that side of things. And, um, he was really interested in how you might apply this to economic decision making as well.

    And, um, I guess the, the overarching goal of this, um, of this project was to, to sort of take these. Phenomena that people don't encode value, um, linearly. And, um, they sort of, it's a very [00:17:00] consistent finding that, um, for example, the difference between one and two pounds seems a lot larger than between 101 and one and two pounds.

    And that's a very, yeah, very consistent finding. And there are various, um, hypotheses about why people might show such a pattern. And then there's a slightly, slightly disjoint set of findings about, um, how people represent probabilities. And they're, the consistent finding is that you sort of overrepresent unlikely events.

    So you think they're much more likely. And that's maybe why people play lotteries, for example. And you at the same time under represent really almost certain events. And so, um, when this project started, it, it just seemed like maybe there's like a really simple set of assumptions we could make that would lead to sort of.

    In a theoretical world would motivate why a decision making system might have these distortions in the value and probability, um, decision making. 

    Benjamin James Kuper-Smith: Yeah, and I mean those, you [00:18:00] said there's like two different things, but they kind of come from the same people or research area, right? It's kind of judgment and decision making.

    It's, it's not like it's completely different fields, right? 

    Keno Juechems: No, no, no. So, sorry. I meant that usually the, when people have tried to, to motivate why people have, have this, for example, value distortion or probability distortion, um, it's usually done separately rather than in an integrated way. Yeah, yeah. Um, so in like a theoretical sort of way, they, they're often a bit disparate, but they are, they are two parts of, of the same problem.

    If you'll. 

    Benjamin James Kuper-Smith: Okay, so the, then the, the kind of origin story of the paper isn't actually quite similar also to the way it's written. 

    Keno Juechems: Yeah, yeah, I think, so 

    Benjamin James Kuper-Smith: this was kind of one thing I was curious about, like, um, you know, you, you write about these two assumptions that you have in here. Um, the, that people, what is it?

    Value, uncertainty. And the other is, God, what's the other one now? Um, oh, it's the, [00:19:00] there's noise 

    Keno Juechems: in 

    Benjamin James Kuper-Smith: the Oh, the noisiness. Yeah, exactly. So if there's two, and I was curious, like, yeah, did this kind of come from someone just like messing around and trying out different things and 

    Keno Juechems: Yeah. So the, the, the noise was definitely at the beginning.

    That's, uh, that's something that Bernie is really interested in, in various domains. Um, so that was really the motivating starting point. And then, yeah, so, and then that's also kind of following how the paper is written is that when you just assume that people are a bit noisy in their decision making, you do get the, the value encoding for free.

    Um, so you, you find that an optimal decision maker that is a bit noisy should encode values, uh, more or less in the same way that, um, we find people to encode them. But you don't see. That there is a similar probability weight function. So the, the optimal decision maker would essentially not differentiate between value and probability and encode them the same way.

    And so then we needed an additional assumption to motivate that. And then [00:20:00] as is assumed in a lot of psychological theories, of course, is that people value certainty and they, they just want to know what the outcome is, whether or not they're, um, they're getting it or whether they're sort of invested in the outcome.

    Uh, and then you, you just plug that in and, uh, you then get this, uh, probability waiting function for free as well. Uh, so it's just these really small assumptions in a sense. 

    Benjamin James Kuper-Smith: So, uh, this is something that just occurred to me whilst you said it, is that, you know, okay, people value certainty, um, certain outcomes of uncertain ones.

    And there's obviously a, it seems to me there's a much more direct link here between that and the probability, uh, distortions. Mm-hmm. Let's say, um, compared to. Uh, people have noisy decision making and value, right? That's less 10. The other one seems fairly obvious, but then it's kind of, it just occurred to me.

    It's kind of counterintuitive that if people value uncertainty, they overestimate uncertain [00:21:00] events and underestimate, uh, very likely, but not certain events. 

    Keno Juechems: Yeah. So you really, you really do need the combination, I think, of the noise and the valuation of certainty to get this pattern. I think we've, we might've tried this without the noise at some point and it didn't come out.

    Um, so then, and either a form that looks like the value encoding or a linear encoding is, is optimal, I think. But, um. I'm, I may be misremembering something here. Um, yeah. But so be one theoretical background to this was that, um, in 2016, I think there was a paper about how the probability encoding might come from noisy and encoding of probabilities.

    So, so to, to backtrack a little bit, um, there's this kind of phenomenon in, in auctions when, so let's say I'm auctioning off a $100 bill or [00:22:00] something, or sorry, like a, a jar of a hundred dollars bills and you dunno exactly how many are in there. Then the final product that you might auction from me, uh, like that you might bid on, is worth exactly the same to anyone who.

    Uh, who placed bids on it, but the person who is most optimistic will win, um, and get, get the money, but they were also likely bit of overestimating compared to everyone else. And that's what's known as the winners curse. And you can make a similar argument for, for probabilities, uh, so that if you, if your system is a little bit noisy, then the, the fact that you chose something might just be driven a bit by your internal noise and you have to correct for that.

    And, um, then they've shown theoretically that one way to correct for this winner's curse in a sense, um, is by having exactly this type of probability, uh, distortion. 

    Benjamin James Kuper-Smith: And the two of them together then lead you to, 

    Keno Juechems: yeah, so, so something [00:23:00] that their theory wouldn't be able to do is sort of differentiate between, so in, in our experiments that we ran, we had, um.

    We had two different groups, and they only differed in the type of feedback that they received. So one group would just get the expected value of an outcome. So for example, just multiplying the value and probability and they would receive that. Um, whereas the other group would actually play out those, uh, those gambles.

    And then, so because the labels were essentially the same and the, the experiment setup was identical, then if you encode probability in one way, then you should do it, uh, for both groups. But we've sort of shown that it really depends on how things are, how things are presented, uh, and whether things are actually risky or not.

    And their, their theory wouldn't have been able to sort of, um, make this distinction between the different experimental manipulations. Um, 

    Benjamin James Kuper-Smith: yeah, 

    Keno Juechems: but it, so, uh, I just wanna sort of highlight that I think the contribution of our [00:24:00]paper is that we have just two very simple assumptions that bring together these two.

    Findings that can be explained in other ways as well. It's not like we've, we haven't come up with a, with a completely new explanation, but I think we've, it's, it's a nice, uh, uh, nice joint explanation perhaps. 

    Benjamin James Kuper-Smith: Yeah. Yeah. It's, it's not like you're the first people who ever explan Yeah, exactly. 

    Keno Juechems: By, not by, not 

    Benjamin James Kuper-Smith: by a 

    Keno Juechems: long shot.

    Benjamin James Kuper-Smith: Certain, like, not even like 30 years of people doing that. But No, I mean, I do like the idea, I mean, I guess that's kind of the, maybe often implicit goal of science right? Is to explain, um, as many things as you can with the simplest assumptions. And this, the simplest model, right? That's kind of, well, I say implicit, but often it's also very explicit.

    Yeah. Um, 

    Keno Juechems: and I think, 

    Benjamin James Kuper-Smith: and so from that, yeah, so from that perspective, I think it's really nice that you also, you know, you also have assumptions that are. I, I think most people would agree with. I don't think there's anything controversial. You know, [00:25:00] it's not like the kind of thing you have, like we have this model and it rests on a pretty weird assumption.

    Um, it rests on assumptions that they definitely make sense, I think from a common sense perspective. Um, but they also just make, especially the noise part just also makes from every scientific evidence thing. I think the certainty, I, I dunno too much. I mean, it's a common assumption everywhere, but I don't actually know how much that's been.

    I know there has been tested, right. With different gambles and that kind of thing. 

    Keno Juechems: Yeah. So there, there's also that literature on, um, on curiosity where, um, so if you, you can even study this with, with monkeys for example, where they might choose to play a certain type of gamble and then they can additionally.

    Sort of pay or wait to see the outcome of that gamble. And they really, really prefer that even though it's, it's non-consequential to them actually receiving it. Um, and you can show the same thing in humans. So there's, there's this kind of element [00:26:00] of kind of wanting to know what you're gonna get. And 

    Benjamin James Kuper-Smith: do you know what paper that is?

    The one with the monkeys, 

    Keno Juechems: uh, by chance? No. Not off the top of my head. Um, it might, but we've, I'm pretty sure we've cited it. Um, monkey 

    Benjamin James Kuper-Smith: Okay. I'll have a look. I mean, I, I'm just asking because I always put all the papers we talk about in the description of the episode so people can, you know, don't have to search through forest Right.

    But can just see what the relevant papers are. Yeah. Um, okay. Then I'll try and figure it out. 

    Keno Juechems: Uh, I can certainly find it as well and send it to you. 

    Benjamin James Kuper-Smith: Mm-hmm. Thanks. Actually you, was it something you wanted to say before I. Something. Okay. Not sure. If not, I'll just continue. Uh, so there's kind of one big picture question I have, which is something that occurred to me whilst reading your paper and related papers recently mm-hmm.

    Is that it seems to me there's kind of, it seems to me there's, there's, in science you always have this back and forth, [00:27:00] right? Someone says something that people say, well, it's actually not that. Right? And then, you know, you go back and forth on these things. Um, and it seems to me, when it comes to opt, uh, kind of optimality, um, and how on rationality in these things, it seems to me that we, we've kind of reached the third level of this conversation now, where at first, the first level is kind of people said like, this is kind of expected utility theory.

    This is how rational people behave. That, you know mm-hmm. That kind of stuff. Yeah. And then the kind of, let's say behavioral economics view then came along and said like, well actually people don't do this. Look, here's all these experiments that show that people don't do this. It seems to me we've kind of now reached the third stage where people say, okay, people don't, people aren't like rational according to expected utility theory, let's say.

    And they do have these biases, but the biases are actually rational once you account for the fact that it's a biological [00:28:00] organism to it and it has certain limitations. And to me at least, it seems like that's kind of where we're at right now, where your paper included, there's this kind of slight, let's say behavioral economics may be overcorrected and now you're kind of correcting back and saying like, well, there's some, there is some rational, it's not completely wrong.

    Is, is that a fair assessment? 

    Keno Juechems: Uh, yeah, I, I think there's certainly a, a good element of it. And that's kind of what I like about this research area is that recently there there's been a lot of, uh, attempts to bring it back a bit to biology and so. I, I guess when I, when I started maybe my master's or something like optimality, like, it, it always sounds so, so final in a way.

    It's like, you know, some things are optimal and others aren't, but really just about anything can be optimal as long as you just define [00:29:00] your, your criterion correctly, right? So I think there's, and it's just that people just assume that, you know, the criterion that people maximize is money or, or some other group that, that you might, um, might look at.

    Um, and that's, that's clearly not how it works in biology. And, um, we, we sort of first have to really think about what is it that biological organisms are optimizing for? And, um, then, then the questions become a bit more fraud and, and intricate. 

    Benjamin James Kuper-Smith: So is it fair to say then that the. It's just the, the, the kind of expect utility theory.

    Um, the people who did that assumed a specific context and thought it was the context, you know, the context of optimizing how much money you make. Whereas if you, as you said, it can be depending on context, uh, lots of things can be optimal. So it's more a matter of Yeah, they just change one context rather than [00:30:00]any of the others that exist or, 

    Keno Juechems: yeah, 

    Benjamin James Kuper-Smith: I don't know.

    I mean, I just, yeah. 

    Keno Juechems: Well, I also think, I mean, it's, it's a very flexible theory, like the, there's a, a huge range of, uh, actual decision rules and functions that, that it can accommodate. So it's, um, it's not necessarily that it's sort of clear cut what, what it should, the final product should look like, but what it, I think we're.

    What it's really highlighting and sort of what I take from it is that what, what people really should be is consistent. So if you're 

    Benjamin James Kuper-Smith: mm-hmm. 

    Keno Juechems: You can optimize whatever and you can have whatever preferences, but I should be able to predict that if you do something in context, A, you should do it again in context a and maybe switch in a different context.

    But there should really be this element of consistency and um 

    Benjamin James Kuper-Smith: mm-hmm. 

    Keno Juechems: I think for, for a lot of economic decisions, whether I want to invest my money somewhere or which job I take based on sort of criteria, [00:31:00] there probably is a fair amount of consistency most of the time. So I do think it, it probably comes close to that, but there still is sort of like that, that lurking background question of, yeah, what's the sort of thing that you might be optimizing that this thing would then give you consistent preferences for.

    Benjamin James Kuper-Smith: Not sure exactly. Answer the last sentence. 

    Keno Juechems: Yeah, elaborate on that bit. Um, I mean, biological systems optimize various variables, right? So you might be like, something that I'm interested in is sort of how can, how do you get from that to having consistent preferences in this super artificial paper money or digital number system, um, where you suddenly have people make broadly good decisions.

    And they might be slightly off, sometimes and slightly biased, but broadly, they're sort of, they're, they're as, as you assume they might be. Um, but [00:32:00] how did the system sort of evolve and what is it optimizing for that that's actually something that's, that's possible that we could, in, could invent it a few thousand years ago to have such a system.

    Benjamin James Kuper-Smith: Okay. So. So where do you, uh, one kind of question I had also when I gave like my very brief overview of, of these kind of three stages, where do you think this is going? I mean, so now we have this more like making it realistic, you might say, or, or, yeah. For, for a biological organism that humans are just making it realistic.

    Is that kind of the end stage of figuring out exactly how that would work? Or is that kind of a, a next, a next stage for, um, the kind of evolution of this argument? You know, like once, once you've figured that out, what, what do you think is, where does that lead us? 

    Keno Juechems: Um, so I, I guess something that, um, that you really want to focus on [00:33:00] probably is you want to be able to.

    Measure people's behavior in certain situations, and you want to be able to transfer that to new situations so you can, you're good at predicting things that you haven't observed yet. And I think we, we can probably do that in like monetary decisions, but I don't think we're very good at, uh, at doing that for more mundane and everyday decisions.

    And I think where, where that, where this whole literature might be going is to focus a bit more on transfer and generalization. And that's certainly where the, at, at least in, in my own sort of view of this, which is of course biased in some ways is where the decision making literature is, seems to be going as well.

    It's like a, a big focus on you observe something in context A and you want to be able to predict something in context B, sort of the stability of it. 

    Benjamin James Kuper-Smith: But in this case where the context are quite different rather than. This [00:34:00] kind of gamble versus this other monetary decision. But do you mean like more kind of saying, okay, how do these kind of gamble findings or how do I, predictions that work here?

    Do they also work for, I don't know, eating? Or do they work for, I don't know what to, I guess, yeah, what job to do, whatever, right? Like just that kind of thing? You mean it's kind 

    Keno Juechems: of Yeah, yeah. For example, 

    Benjamin James Kuper-Smith: yeah. 

    Keno Juechems: Or like maybe there are, so people might have measured people's utility functions and then used to predict whether they want to place some risky gambles or whether they want to take out insurance and that kind of stuff.

    Uh, and that might work quite well, but maybe there are, there are even different types of measurements that we might take maybe in completely different tasks that better describe how people will behave in, um, in other situations. 

    Benjamin James Kuper-Smith: Mm-hmm. 

    Keno Juechems: So I think it, it's a, it's often a question of like, well, what do you want to use it for?

    And like if you, if if you just work, say in the financial industries, [00:35:00] you probably don't need to need to worry about this too much. Uh, 'cause your, your models are sort of approximately fine. But if you, if you're interested in how all this is implemented in the brain, then I think it mm-hmm. It matters quite a lot.

    Benjamin James Kuper-Smith: Right, right. Yeah. Is, uh, this kind of relates to something I just wanted to ask, which is kind of what, why are we searching for optimality or deviations of optimality? What exactly is the point here? 

    Keno Juechems: So, I sort of alluded to this earlier, is that, um, I think we, you should be careful with calling something optimal or appealing to optimality, but what I think it's very good for is to, from a sort of modeling and understanding perspective.

    So if you have, if you have a model that makes certain assumptions about what is being, well now I wanna say optimized, but something that's, uh, that it's trying to achieve in a way, uh, which is I guess the same way of saying it. 

    Benjamin James Kuper-Smith: Gotta say 

    Keno Juechems: that. Yeah. 

    Benjamin James Kuper-Smith: Um, 

    Keno Juechems: then, uh, then you can sort of see where, how [00:36:00] far that theory can push you and, um, use it to, to make predictions about where you should place your next experiment or maybe look for brain signals, you know, anyway.

    And that's, I think where, um, appealing to some sort of optimality is good for. And, and I think we, we implicitly, we, we do that for most aspects of science. Um. But we, we may not like as, as, as soon as you're using some kind of model, there is some implicit appeal to, to optimality sort of baked into it. 

    Benjamin James Kuper-Smith: I mean, it seems to me in the, you know, in physics, let's say if you have a e theory of gravity that doesn't optimality, like it seems to me like there might not be a con, you know, you don't go, is the gravitational system optimal with response?

    Right? Right. You just go like, how, how accurate am I describing this? Right. So is it, is the difference here that the, the thing that your modeling [00:37:00] has a, is trying to do something rather than just existing in that sense? Is that the difference here or, 

    Keno Juechems: yeah, I think, I think part of the difference is that you have a system that makes decisions and it makes decisions to some end.

    Um, and 

    Benjamin James Kuper-Smith: right, 

    Keno Juechems: and usually just, um, just having descriptive theories are, you know, they're. They're nice, but they, they usually don't make, um, very strong predictions about, about situations that are outside of the things that you've, you've looked at. And for that, I think it's 

    Benjamin James Kuper-Smith: mm-hmm. 

    Keno Juechems: From a sort of understanding perspective, I think it's really, I, I, I tend to personally think of it as like a, a tool for understanding more so than necessarily saying that this is exactly what people like, that there is some, some wheel in their head that counts.

    Um, like, am I really certain or uncertain? Um, yeah. So it's, I think it's like a, a epistemic sort of tool. 

    Benjamin James Kuper-Smith: Yeah. And I mean, [00:38:00] that's also kind of what you wrote a few times, I think your abstract and the end of the significance paragraph, this kind of idea that, you know, actually having some sort of causal explanation rather than a mere description.

    And Yeah, I agree. Like descriptions are perfectly fine. That's also how you start, right? Yeah. Uh, you don't start by. Causally describing something before, you know, what you, what you're looking at. But yeah, I guess, I guess that is kind of the nice to maybe slightly maybe close off this part a bit. I think what the nice thing about this paper is that yeah, you have these two fairly basic assumptions that aren't even particularly specific to the context 

    Keno Juechems: Right.

    Benjamin James Kuper-Smith: Um, necessarily that you're looking at. And that through that you can achieve this kind of, not exactly cause of understanding, but it's, it's a potential explanation of these things that people have been talking about for decades now. 

    Keno Juechems: Yeah. 

    Benjamin James Kuper-Smith: Yeah. I have to ask one question [00:39:00] though, which is something, something I was curious about and I think it's one of the last sentences you have.

    It's actually the very last, maybe I'll just read this. Nor does our theory consider the most distinctive contribution of prospect theory, the intuition. That computation of value is reference dependent with all utilities evaluated relative to a status quo, given by the current context. The normative properties of such reference dependents, for example, in the context of efficient coating and efficient computation have been discussed elsewhere.

    So I haven't read this paper by zos, but just briefly, does your, so I'm ki partially interested in these kind of losses and gains things. 

    Keno Juechems: Mm-hmm. 

    Benjamin James Kuper-Smith: Can your paper say anything or your, your kind of I model say anything about that? Or is it really just this only works in the, in the gain context? Don't ask me about losses.

    Keno Juechems: Um, yeah, I mean, I haven't looked into the, the loss framing. So I'm not sure what the model would predict [00:40:00] there, but I, I don't see any reason why it, why it shouldn't be able to at least, I mean, it is essentially just the, the same function. It's just flipped. The only thing that it doesn't, that, that I'd be surprised if it really fit was whether people are more loss averse, like have a, have a stronger coding for losses than they do for against.

    So, uh, I'm, I don't think that the model would necessarily do that. 

    Benjamin James Kuper-Smith: So you're assuming, uh, symmetric? 

    Keno Juechems: Yeah, I mean, for, for the purposes of the paper, we were sort of, uh, assuming that there are no losses at all. Um, so, um, so I guess we're sort of, uh, avoiding that question a little bit. Um, but I, I mean, there, there's been a.

    I mean, there's ongoing debate about how much of a thing loss aversion really is. 

    Benjamin James Kuper-Smith: Yeah, definitely. 

    Keno Juechems: So it, it may not be as strong as we might have assumed a few decades ago anyway. And the sort of, I think the more general point is that, [00:41:00] so just by saying that there are gains and losses, you are already assuming something like, um, like prospect theory because losses are actually also not defined in, um, an expected utility function in its sort of earliest incarnation.

    So that, that is an additional assumption that we've sort of just implicitly made. And your, your reference point might switch and like turn losses into gains and vice versa, um, in various interesting ways. But we're just not looking at it. We're just assuming that there is a reference point that you're using.

    Benjamin James Kuper-Smith: Yeah, I mean, so one reason I was asking about this is because this is something I'm kind of, this is kind of, I guess like meta science, how to, how to do science and also like what a paper exactly is because, so this is something I've been, so I did, I did basically the first thing I did with my PhD and it's, it's not out yet.

    Um, and we still wanna do like at least one experiment, but it seems right now that we have, we don't exactly, we kinda did a different decision making [00:42:00] context, a more social context, and we find not exactly the, the traditional loss aversion definition in terms of different slopes for positive and negative values.

    Mm-hmm. But people definitely try and avoid losses in this context. So I had this kind of very simple way of testing this where, okay, something could be a gain or a loss. And I had this like, very neat way of doing it and I was very happy with that. And then, but it avoided zero because zero in this context was difficult to interpret and mm-hmm.

    Uh, in the way that set it up, it made a bit awkward and I thought, well. I'll just, I'll just do gains and losses, right? Like, don't care about zero right now. I'll figure that out later. But then at some point, as I was, you know, analyzing my date and thinking about this, like, are you stupid? You can't do a theory about like, how people behave in this and just avoid the concept of zero.

    Like it's a, it's a real thing. You can't just pretend it doesn't exist. But it seems to me that's kind of exactly what you said you did. Right. And my question there is kind of like, I guess the general question is kind of like, [00:43:00] that I'm struggling with is like, when is a paper finished and how much does a paper have to achieve?

    Keno Juechems: Right. 

    Benjamin James Kuper-Smith: Um, because it seems to me that's saying like, if you do something that is about, like gambles and things, you know, working out to different degrees, um, in terms of how much money you get, it seems kind of so incomplete to not include nothing happens or you lose money. 

    Keno Juechems: Mm-hmm. 

    Benjamin James Kuper-Smith: Um, so yeah, I'm just curious like how you think about that kind of problem.

    Keno Juechems: Right. So like. I guess the, so we're implicitly assuming, I guess, that the zero point for people is just however they come into the experiment and then, um, like nothing happens would be one of the possible outcomes of, um, of the gamble. So in one of the groups at least, um, we did have that. Nothing happens, but yeah, the more general point of like, when is a paper finished?

    I mean, I think that's a, yeah, that's, that's very difficult. I [00:44:00] mean, um, that's 

    Benjamin James Kuper-Smith: why I'm asking you and not vice versa. 

    Keno Juechems: Um, I think it, I mean like when you, when you have a paper, I think it, it should make a, an interesting point about some phenomenon and ideally also include maybe some, some modeling where you might, uh, sort of.

    Again, have a more theoretical understanding of, of what you're looking at. And if that's then also surprising, then maybe it gets into, uh, into higher journals because that's sort of what they reward, I guess, whether something's surprising or not. But I personally, um, don't think that a paper really, like, as long as you're not writing a theory paper and it's still about sort of empirical data, I don't think it needs to look at all the different aspects that you might also look at, I guess, um, from one theory.

    So I think we've, [00:45:00] we've sort of just taken the, like the, the sort of basics of the problem and just look at those. And then you can do a follow up study. We might do, I mean we don't have very specific plans at the moment, but there, we probably mentioned those in the discussion that there are like various ways in which you could extend this.

    And I think like. Yeah, having the solid base that you can sort of work off is, I think then worth publishing. Um, and then either you or other people can, can work with that because otherwise you, you'd probably just end up having huge projects that either take years to completion or maybe never get finished because you also have to do other things and probably need, like, sadly enough, you also need some papers like fairly regularly.

    So there's, there's definitely an incentive to maybe be a bit more incomplete.

    Benjamin James Kuper-Smith: Yeah, that's, that's another problem where I [00:46:00] basically have like these few experiments that are kind of smaller experiments we did and they seem to kind of work out. And then I realized like, well, I can actually generalize this too much broader way of testing it. Um, so not like so specific to this one task, but it's a much more general way.

    But then I realized like, well. If I do the second, the first one basically doesn't matter anymore because the second one kind of includes the first one in some sense. So like, do I even need to publish the first thing? Like, because you know, if I publish that and it replicates and the other one, then it's like, okay.

    Mm-hmm. 

    The second one by itself would've said that if the first one doesn't replicate, then you know, you've created a star around something that doesn't work, so however long it took you to finish the second one. So, yeah. But then I also, you know, I, I also know that, yeah, if you don't publish stuff, you're not gonna get a job afterwards.

    So it's, 

    Keno Juechems: yeah. 

    Benjamin James Kuper-Smith: Yeah. I had a really difficult problem, 

    Keno Juechems: but I think there's also a. Like, it's also a communication aspect, I think, in that, like, you might go to conferences and talk to [00:47:00] people, but really the major way you communicate with other scientists is through your papers. So if you, um, if you have a, something that, that you believe in and you've sort of made sure that, you know, you've ruled out other alternative explanations and stuff, then it may be worth publishing just for that aspect of it, that you're, that, that other people then know about it and, um, they can take it and do it, do whatever they want with it.

    Um, even, and then maybe it doesn't replicate in your own work, and then you can say that later on. Um, and I guess that that's just what happens. Like, no, not everything will replicate. 

    Benjamin James Kuper-Smith: Yeah. And I guess, I mean, yeah, it's not like we're, you know, doing something. And then, you know, ping or putting Yeah, exactly.

    In there, you know, there, there was a, like, initial thing that we wanted to do and it seems to work out, so it's not like, yeah, we're, we're presenting exploration as hypothesis driven research, but, um, yeah, it [00:48:00]just feels, it feels so, yeah, I know, like I'm not, it feels a bit like I'm not doing a proper job in that sense, you know, present the, like, present it once you're finished Right.

    Rather than mm-hmm. But yeah, I guess, 

    Keno Juechems: yeah, I mean it's, it's, it's hard to say from like, as a, as a sort of general point Yeah. The abstract. But, 

    Benjamin James Kuper-Smith: um, 

    Keno Juechems: yeah, if you, if you think that what you, what you have at the moment makes, makes a valid point, then you might think about publishing it now. Or if you think that maybe in a year's time you have something more general that makes a, uh, a valid point that's bigger than this, it would 

    Benjamin James Kuper-Smith: make the same point.

    But yeah, 

    Keno Juechems: but also from a practical, practical perspective. So de depending on. Or what the point is that you're making, if you have something that already went through peer review that you can point to that might reinforce your people might be more inclined to, um, to take it, um, take it seriously if there's already published evidence on it depend, [00:49:00] depending on how no.

    How novel it is, but it might be a proper, 

    Benjamin James Kuper-Smith: yeah, I don't know. Novelty. And I have to admit, like, I think when you are, especially when I was in my Bachelor's of Masters, it seemed always like, you know, people have these findings that come out of nowhere and then once you, you start doing the field, you realize like, eh, that already existed.

    Why is this in like, you know, psychological science or whatever. Like why is this such a big deal? So, yeah. Uh, I think I still have a lot to learn about. Academia as a, as an enterprise rather than science per se. 

    Keno Juechems: Yeah. And I mean, the novelty is like, you have to ask seven novelty to whom, right? It's like something may be very well established, like some, some methods, for example, like there are in neuroscience, there are a lot of methods that now come from physics, um, or even pure mathematics.

    And if you showed this to a mathematician, they'd be like, well, we'll die. I already know this. And it's been proven for many years. Yeah. But, [00:50:00] uh, neuroscience is still like, it hasn't been applied yet and people still need to sort of know about it. So I think it, it's really easy to, like, when you're so deep in a topic, um, and you, you read widely, then it might seem that something, you know, that it is, uh, that it's maybe been around for a while, but that may not be the case of, of a lot of other people.

    So it might still be worth, 

    Benjamin James Kuper-Smith: yeah. 

    Keno Juechems: As long as you, obviously you shouldn't, you shouldn't then sell it as being something very novel. But that might explain where, where some things sort of get, um, get rehashed, uh, a few years later. 

    Benjamin James Kuper-Smith: Yeah, I mean that's always the kind of classic thing, right? You think you have this new thing and then there's 10 papers from the sixties or something that, uh, of incited like 10 times and they kind of already did what you wanted to do.

    Although I like finding those papers. It always feels like a, you found something hidden that no one knows about. 

    Keno Juechems: Yeah, exactly. 

    Benjamin James Kuper-Smith: Yeah. Anyway, I think I'm, shall we, uh, [00:51:00] end my therapy session and start talking about your other papers? That's 

    Keno Juechems: true. 

    Benjamin James Kuper-Smith: So very dramatically. I'll turn the paper so I have it in front of me.

    Um, so the other papers, where does value come from and can you maybe explain what exactly the problem is with reinforcement learning that you're highlighting in this and. And if you can do so, maybe also explain what reinforcement learning is, but maybe we have to do those in two separate steps. 

    Keno Juechems: Sure. So I guess the, the problem that we're interested in here is where, um, an agent, so for example, a person or an animal interacts with, with an environment.

    And, um, so they, they do something and then they see how the environment reacts to it. So for example, you might, um, might go to your kitchen and eat something. And then, um, what in reinforcement learning is classically assumed is that you would then, um, or have an observation, which is sort of like the world has [00:52:00] changed after you've acted.

    So what does it look like now? And you also get a signal that you might call reward, where it's sort of telling you how good or bad it was that you've just done. And both of these things, at least classically come from the environment. So it's something external to you. And then you take this, um, this reward signal and you want to get most reward over a long period of time.

    So you, you're sort of like a, a rational agent that wants to maximize its reward or happiness or whatever else you might want to call it. But then the problem that we're, we're highlighting and is that at least in psychology and neuroscience, we often assume that we know exactly what that reward is. So it, it is fairly straightforward.

    If like people are gambling for money, like in our recent study, then it's probably fair to assume that this is what they're interested in maximizing the, the amount of money they gain over the course of the experiment. But things are a lot less clear if you include [00:53:00] things like, um, social decision making or even just things like when you decide to eat and when you decide to drink something or when you decide to exercise.

    And. And so on. Um, so the problem that we're interested in is like what an agent or a biological system has to do is it sort of takes the observation from the environment, but it then sort of splits that into this is how the world has changed, but also something that I can use as a learning signal to make better choices in the future.

    Benjamin James Kuper-Smith: So I think if you've already alluded to this, but I don't think it is, maybe not say exactly the, what you call the reward paradox between this kind of like the reward is out there. But then it also kind of has to be generated internally. Can you explain that a bit? 

    Keno Juechems: Yeah. So in, like I said, in in most scenarios where when reward is generated and when the environment gives you reward is [00:54:00] either given by the experimenter.

    So exactly when I pay out money or, um, when you're teaching a, an artificial system, uh, it receives these numerical rewards once in a while when it does something good, uh, like for example, a self-driving car, when it just, when it doesn't veer off the road, then it gets a reward, for example. And, um, in, in neuroscience, we might, for example, if, if we're doing experiments in rodents or.

    Monkeys then, uh, they would be in a state where they're actually, where we know they might be hungry or thirsty, so that then giving them a food or drink reward is rewarding to them, which it otherwise wouldn't be if they weren't hungry, for example. So we're deliberately putting PE people or animals in that position where that we're fairly certain that that's what they, um, they're interested in, but we don't really consider that.

    Well, that's also a kind of decision that our system has to make by itself. Like it sort of [00:55:00] needs a, needs a separate layer to the, to the whole learning problem where you're using that reward signal to, um, to teach yourself how to behave better. But you're also sort of in the first place, you need to decide what is it actually that I want and.

    That's probably easy for things like hunger and thirst, like, you know, our, our bodies tell us fairly explicitly that that's, that's what we should be doing. But it becomes a very difficult problem if, like, for things like, you know, what career do I want to pursue? Or things like that. And we, we just wanted to highlight that in, even in sort of fairly basic problems where you might be pursuing two goals rather than just one goal.

    There. There's already a lot of, a lot of interesting data that we might want to look at that, um, where, where there's, where people might deviate from, from sort of classical assumptions. And again, we're sort of then at different bits of economics that have looked at that, of course, over, um, in the sort of aggregate.

    But [00:56:00] we just, like with this paper, we sort of wanted to highlight that that's not being considered as much in psychology and neuroscience at the moment. 

    Benjamin James Kuper-Smith: Yeah, I mean, this is something I'm really interested in because, you know, I do social neuroscience, so this kind of, this social aspects are so, you know, people's motivations for, for behaving, let's say in a classic economic game.

    You know, there's always all these different considerations that go into it. So that, that's kind of why I found, you know, I think also why I start, why I read this paper is just because it kind of has this, um, I'm really interested in this. How do you behave if you are, you know, you're not just trying to make money from a gamble thing where basically that's the only thing you're trying to optimize.

    But what you do, if you have, you know, you're interacting with someone, you, let's say you know them or only a bit or something, you also wanna make money, but all these kind of things that suddenly flow into that, um, right. That's something I'm really interested in. And maybe can you [00:57:00] then, so it, it seems to me kind of your.

    Tentative suggestion or maybe a bit more than tentative suggestion is to use this thing called homeostatically regulated reinforcement learning, or that is a theory that tries to, has tried to address this or something. 

    Keno Juechems: Yeah. 

    Benjamin James Kuper-Smith: Um, I am not really, I haven't been familiar with this outside of what I've read from your paper, although I'll certainly read papers from that soon.

    But what does that exact, what does that add to reinforcement learning and how specific is it to homeostatic homeostatic processes? 

    Keno Juechems: So the, the specific bits that we talk about in the paper are taken from, um, from a paper by media Kara and Boris Goodkin, who's sort of taken the decades or literature on homeostasis and sort of tried to stick that into reinforcement learning.

    And so this is essentially just providing a way in which. [00:58:00] Let's say have a goal, which might be, you know, I don't want to be hungry, I don't want to be thirsty. Then, um, you can easily use the system that they've described to generate rewards that then you can learn about as a, as a reinforcement learning agent.

    And they've shown that in, in a lot of different scenarios. Um, it accounts for empirical data from, um, I think they exclusively talk about rodents. So we're sort of saying that, well, maybe if you sort of take the essence of this as like, there are probably gold state states in the world, like you want to finish your PhD.

    And that sort of decomposers into, you know, I want to publish this paper and then work on that paper. And how do you then use that knowledge to, to motivate yourself and to teach yourself whether something was good or bad. So we're sort of saying that, well, it's. It's essentially a, a comparative process.

    So you have like a, an understanding or like a sense of where you are and a sort of [00:59:00] sense of your, your goal state and you're, you're trying to measure the distance between them in some, some arbitrary way. And then as long as you decrease the distance to that, you're rewarding yourself. And if you're moving away from it, you're not rewarding yourself, you're maybe punishing yourself.

    And, um, so we're sort of using this framework that exists for sort of low level decision making and saying maybe we can take it also to more abstract sort of decisions. 

    Benjamin James Kuper-Smith: And there's nothing inherently that makes, I mean, I guess you suggested it as a general kind of thing, not just something that works only for specific kind of decision processes, right?

    Keno Juechems: Yeah, I think it would, it's fairly general. So the way we've pre what we've. Presented in the paper, which is like strongly based on the Kara Goodkin work sort of assumes that your, your set point or your goal is always fixed. That's obviously not the case for a lot of interesting problems. Um, and so the goal posts are constantly evolving [01:00:00] and, um, so that would of course require quite a bit of extra work to make that fit.

    But I think the sort of what I liked about their framework when reading about it and when we've already collected data that sort of, that spoke to this, um, and this also something we talk about in the paper. I just think that it's a really important process that we should think about. Like what is the, what is the reward generating process and.

    What are some properties of that that we might want to measure? And I think that's something that really, there hasn't been all that much work on. And um, it's certainly an active area in, um, in like machine learning in, in some ways where you might teach your system to, to learn from experience that maybe going through a doorway is a good idea or something if you want to explore an area.

    But I think we've, yeah, we, we don't know a whole lot about this phenomena and it really, to me, like something I wanted to highlight with the paper is really that that is a process that we really should think about more. [01:01:00] Uh, especially if you want to talk about more human like phenomena. Like, you know, you're, you feel satisfied when you reach a goal or you feel disappointed if it becomes unavailable.

    Um, and then sort of like, yeah, asking yourself where, where does that that signal come from? How does it originate Then it. That's maybe not quite easy to figure out. 

    Benjamin James Kuper-Smith: Yeah, I mean, because, um, I did have a slight, one thought I had whilst reading is just like, what does this actually add that isn't already in the hemostatic ir regulated reinforcement learning framework?

    Is it, is it this kind of combination of saying, look, people are doing this thing in this context, but we should also do it in this context. Do, do you see kind of what I mean, because at some point I thought like, but they've already said what's in the home static context. Like, I mean, for me it was useful just to, you know, obviously invited you because it was an interesting paper because I thought [01:02:00] it was pointless.

    And, um, but do you see kind of the, what I'm trying to get at, 

    Keno Juechems: uh, yeah, I do. Um, so I mean, sort of the, the way this paper started was as a kind of discussion of the work that I'd done during my. PhD. So that's sort of like we, we sort of wanted to have a, something that summarizes the findings from my, from my PhD, and those are, those are sort of heavily based on, um, that there might be certain goals that people are trying to achieve.

    And so these are the, the studies that we talk about. And then later on we sort of found that the framework that they have for homeostatic, uh, decision making actually might fit this quite well. So it is, it is right that the, the framework that we're describing here is essentially there the sort of mathematical framework, but it is a very, it's a very different type of problem that we're interested in that they don't consider at all in their [01:03:00]paper.

    Okay. Um, so it's like, yeah, again, sort of taking that, which, which you rightly say existed, but um, combining it with the work that we've done that might speak to it in, in the different domain of human decision making. 

    Benjamin James Kuper-Smith: Mm-hmm. Uh, so still if you kind of, they, they were kind of general but not super linked.

    So I'll just ask them probably as a list. Mm-hmm. Uh, the next few minutes. Um, one question I had was, so you have this, so you said, okay, we, you know, the question is where does the value come from? And then you say, okay, we have these goals. We're trying to minimize the distance to it. Isn't that in some, so, so now the problem is where do goals come from?

    Right? And I think this is also a header you have in here, right? Um, or something like that. Um, I mean is to, another kind of critical question I had was to what extent is this not just [01:04:00] substituting one mystery with another, or is that just what we do in science? 

    Keno Juechems: I. So, I mean, sure. It's a, it's a different, uh, like it's a very difficult problem to think about, um, you know, where are these, these golds coming from.

    But I do think it's a, it's a separate, uh, related problem because you can, you can then have a system that tries to come up with good golds, for example, exploring your environment in certain ways. And then you can ask what is the, the reward signal that that system is generating to teach an agent how to behave well.

    So you've sort of, you can then look at two different aspects essentially of the same problem and interrogate whether maybe those signals that are generated by your goal system are reflected in some lower level brain areas. And, um, as a sort of second point, I think, I do think that these map onto different.

    [01:05:00] Brain areas. So I think that the, um, reinforcement learning problems that, um, we might study in, in animals, they're often primary reinforcers, so food or drink rewards. And it's quite, quite likely that these are fairly hardwired systems that rely on brain systems that are fairly ancient. But then we also have this, this whole lump of brain in the front of our head that sort of, um, is the most expanded compared to, to other primates and humans.

    And, um, this is, that's the sort of area where we find this like, uh, longer temporal contexts and, um, reversals and control that's, um, switching be between different tasks that we might be doing. Uh, so I do think there's, there's at, at least that's sort of what I, what I think might be the case is that, um, thinking about things as sort of like higher level states or goals is more likely to explain.

    [01:06:00] Activity and prefrontal areas, whereas something that's like fairly low level, like rewards might be more able to explain things like the basal ganglia, for example, where, where you might have in investigated that. 

    Benjamin James Kuper-Smith: But So another question related to that is isn't, it seems to me that it's kind of circular though, right?

    In the sense like, okay, why do you have goals because of rewarding? So like why would you have a goal if it's not rewarding? So then, yeah, 

    Keno Juechems: right. I think that's, I mean that, that's true if you just have one goal. But if as long as you have competing goals, then do you can ask yourself do they compete as a sort of pre reinforcing state or do they compete at the state where you're, where you are reinforcing your actions?

    But I don't know, I mean that's like, I guess like the, the sort of brain starts with like fairly simple mechanisms, um, evolutionary speaking and then it sort of adds, adds new, new layers to it. And I think. All we're really saying is that [01:07:00] yeah, there's, there's, there's a new layer to it in a way. 

    Benjamin James Kuper-Smith: And these are the kind of problems that they might try to solve or it might be involved in solving.

    Keno Juechems: Yeah, so I guess like, it is also a question of granularity, I guess in the sense that, um, if you, if you take classical reinforcement learning, then rewards are defined as a sort of state by state kind of signal. So on every single step you might get a reward signal or um, something like that. But you might want to have a sort of, on top of that, you might want to have different layers of abstraction that sort of decompose the problem into, I don't know, rooms or like sub goals, like writing a paper or something like that.

    And, um, we know that there certainly is something that, that, that the brain certainly does that in, in various domains. Um, but I guess we're, the specific point we're making is that it also like that is where you generate your rewards from. It's not at a lower level. 

    Benjamin James Kuper-Smith: And I [01:08:00] mean, so I'm assuming that this is something that's not that well studied.

    Um, but one thing I'm curious about is that, okay, so you have, you know, you have a goal that's, uh, consists of multiple dimensions. Say, let's say it's just two, right? As you do in your paper, how do, so in the, in the, I think the paper, if I remember correctly, you, they're linear related, right? Both are equally weighted and that kinda stuff is, has much been done on what happens if you have multiple goals that have different importance to the person, um, and these kind of things.

    Keno Juechems: Uh, yeah. So that's, so if you just look at decision making, um, then there's quite a large literature in, in economics that would look at. They might call it indifference curves or rates of substitution where you might have a good, uh, good A and a good B, and you have different preferences according to those.

    So you might be willing to give up five units of [01:09:00] A to get one more unit of B. So you sort of lie on these indifference curves where the whole, the whole bundle of things is still worth the same, but you might require that you get a large amount of one good for giving up a small amount of, of another. So there's definitely a good literature on that.

    And there's another, uh, sort of more psychological literature on if you, if you have things that if have different attributes. So I don't know your, um, I know going on a holiday you might think about, um, the hotel, the city and, and so on, and you might want to integrate that into a single decision. But for.

    For what we are interested in is sort of like where not all of these attributes will factor in, in the decision that you're making. So it might be that you, you have these discreet things that you collect that are only, only rewarding in one dimension but not the other. And so you're like, you're, you're continuously changing, changing the state that you, you're in or the amount of resources that you [01:10:00] have, which is, uh, a bit different from like the multi-attribute sort of view, but there's certainly, uh, literatures that will probably have good links to, to this.

    Benjamin James Kuper-Smith: Yeah. I need to, I really need to read much more about reinforcement learning. It's like one of those topics I keep kind, I want to get into, but then I keep like, you know, doing these other studies. Um. 

    Keno Juechems: I mean, it's incredibly vast. So, um, 

    Benjamin James Kuper-Smith: yeah, 

    Keno Juechems: it's very easy to get 

    Benjamin James Kuper-Smith: overwhelmed. That's actually, that's a very good link to what I want to talk about next.

    Not exactly being overwhelmed, but, um, on a kind of general level about doing a PhD. 

    Mm-hmm. 

    Uh, one thing that I'm still, I, I dunno how to solve and um, it's obviously no, I'm assuming maybe you have it, but I'm assuming there's no one fits all solution. Um, if you have it, feel free to say it, but like, one thing in terms of that I especially have is that it's just how both in sense, like is it possible [01:11:00] and also in sense of like what other steps and practically how would you do this?

    How am I supposed to learn all this stuff? 'cause you have this, you know, if you wanna do anything interesting, you have to. Not necessarily have to, but you combine different things that haven't been combined before it. Mm-hmm. So in my case it's, you know, if you have social decision making on neuroscience, I'm interested in the kind of game theoretic aspects.

    I am, you know, interested in the kind of cognitive and computational neuroscience things. Yeah. Um, there's also this whole evolutionary biology stuff. There's, you know, basically it seems like most disciplines do game theory in some way or another, or it relates to it. This is leaving out all the technical aspects to program learning how to do all the open science stuff, all of that kind of stuff.

    So I'm just curious like. For example, how do you go about learning reinforcement learning, maybe as one example? Like is it, yeah. Other than reading your paper, of course. 

    Keno Juechems: Um, yeah. I [01:12:00] mean, you just have to remind yourself that whatever topic you're studying, you probably can't read everything anyway. So there's definitely like free yourself of the illusion that you haven't read everything yet because you haven't, it's impossible.

    Um, and that's particularly the case for something like reinforcement learning. That's like probably more papers published in a day than I could read in a month. So yeah, you, you just have to have to like get yourself to a state where you feel confident that you can run your experiment. I think, you know, you, you don't just stop reading and learning then.

    Right. But there, there, there probably is a point where you. If, if it's very new to you, you might be a bit unsure about it and you just have to, I guess, trust your supervisor that they, they get a sense of, is this going to be novel and well motivated for the community that you want to address this [01:13:00] for?

    Benjamin James Kuper-Smith: That's almost the hardest part for me, though. Like, I don't know what community I want to address. 

    Keno Juechems: Yeah. 

    Benjamin James Kuper-Smith: I just wanna do my thing. That's the kind of, 

    Keno Juechems: yeah, and I mean, that's, that's good. I mean, you'll, you'll find a place for it in the end if you, if you do what, what you're interested in. And that's definitely the main thing, I think.

    Benjamin James Kuper-Smith: Yeah. And it is also more confined. Um, I think, you know, I mean at some point we did think like, is this what we're doing relevant? Economics or something. Right? Because I've been reading a lot of economics papers over the last two years and, but at some points you realize like, no, they won't care. This is, and I actually had someone who knows a bit about the field tell me that's like, they won't care about this.

    Like this is, this is trivial to them. 'cause they don't, you know, they have different goals and aims and questions and that kind of thing. But I'm curious actually on a practical level mm-hmm. Like how do you, let's say you want to learn something new, new, let's say you say, okay, I, I don't know, you want to get better at probability [01:14:00] theory or whatever, you know, something that's not, it's direct, it's related to what you're doing, but it's not something you're gonna learn on the job per se.

    Keno Juechems: Yeah. 

    Benjamin James Kuper-Smith: Might have these kind of long term benefits. I, I keep like going back and forth whether I should do it in a block and just like, take a course mm-hmm. Online or something and just do it in like, you know, two weeks and do it once or to kind of spread it out over like half a year or a year. Because then, you know, both have kind of advantages, disadvantages.

    Keno Juechems: Yeah, yeah, no. So if it's something quite mathematical, I personally find that quite difficult to just learn on my own, um, or just from, from reading and doing my own exercises. So for that, I would definitely recommend doing courses if you can. And, um, for me personally, it would make more sense to do that as a, like, as a block or like a maybe one or two month kind of thing.

    These things also take a long time to learn, so you [01:15:00] will have to invest, uh, a good amount of time. But, um, something that definitely hasn't worked for me was. Doing a little bit of it two hours a week and then I do it for a couple of weeks and then it sort of fades into the background and then I don't get back to it.

    So, um, if you're anything like that, just do the block course. And that's also something that's really difficult to do once you're no longer, uh, sort of a student. Um, 'cause then you, you know, now you can, you can rely on all the resources at the university and go, maybe even officially enroll in some courses and afterwards it becomes a bit harder to do these sort of fairly mathematical kind of things.

    I think 

    Benjamin James Kuper-Smith: we could still attend them, right? Or I don't know. Maybe depends 

    Keno Juechems: on the country. It depends. Maybe, I mean there's also enough, enough online courses that, that you can do. But I, I guess what really helps with this, like doing the actual physical courses is that you would then be able to go to tutorials and you [01:16:00] have exercise sheets and that's kind of necessary I think for.

    For something as practical as learning how to do certain mathematical things. 

    Benjamin James Kuper-Smith: Yeah. Have you had to do lots of that? I'm just curious. Like for, because I guess like the, yeah, like for example, your PNAS paper, there's, you know, there's some mathematics in there, but none of it is exactly well changing. Yeah.

    But I'm curious like how much you need to know to do that kind of stuff. Is it, you know, some basic algebra and then you're good to first or, 

    Keno Juechems: um, it de, 

    Benjamin James Kuper-Smith: I guess it's also the question like, like how much do you have to learn about something? Right. 

    Keno Juechems: Yeah. I guess it, it depends a bit like you're, if you're interested in, in the human behavior of it, then you probably don't need to worry too much about it.

    Um, because then you won't be someone who's like coming up with a new theory for which you probably have to. [01:17:00] Lots of proofs and, and so on. Um, so I think for, for everyday work, like a good basic knowledge of calculus, algebra, probability theory would probably get you a long way. But then for every, for every kind of project that I did, I did have to read up about some specific things.

    So for example, for this one, I had to read more about various types of probability distributions, for example, the gumball distribution that I hadn't really encountered before either, but it was, turns out to be quite important to it. And, uh, yeah, but you, I mean, I don't think for psychological research you really need that background.

    That's certainly helpful if you have it, I guess. But 

    Benjamin James Kuper-Smith: yeah. So, so was it for you then, a kind of having a, a base knowledge of those three areas you mentioned and then advancing as needed on the job or, 

    Keno Juechems: yeah, I mean, in, in some, in some ways I sometimes would've. [01:18:00] Wished I had a better formal mathematical background.

    Especially when you're, as I'm sure you're doing as well, if you're reading economics papers, they're quite, you know, they're quite mathematical and maybe not always written in a way that's very reader friendly. 

    Benjamin James Kuper-Smith: Yeah. Yeah. 

    Keno Juechems: So for those, I I, I often wished I had a, I had a stronger background in either the maths behind it or the formal economics.

    But yeah, it's a, a good base knowledge gets, gets you quite far, I think. 

    Benjamin James Kuper-Smith: Yeah. I mean, yeah. One thing I find just really interesting is that, for example, we did this linear algebra course that we kind of organized for ourselves. Mm-hmm. So we had this like MIT open course where stuff and used that and then.

    We did this once a week, and then we got about halfway through, and then people went on holiday and everything fell apart and started again. So that's exactly the way you described it. Um, and actually that worked fairly well, even though we didn't have anyone who kind of knew [01:19:00] what, what they were doing helping us or anything.

    But for example, what I found really interesting is, you know, we just did this further than I needed to really for what I was doing. And then suddenly, you know, then you realize a few months later it's like, oh, this is how this thing works. Like, I can explain this thing now. I didn't even know that this was what it was, but now that, you know, you know, it, it's, these things become so much easier.

    And then, so since I had that realization, I think like, ah, maybe I should learn as much as I can about this. 

    Keno Juechems: Yeah. 

    Benjamin James Kuper-Smith: But I guess, I guess there's again, the practicalities, right? 

    Keno Juechems: Yeah. I mean, I would certainly recommend doing that, but it is very time intensive. So there's, there's a trade off. 

    Benjamin James Kuper-Smith: Uh, it seems like you don't have a one fits all.

    Keno Juechems: I don't. 

    Benjamin James Kuper-Smith: God damnit. Okay. Um, um, maybe as a kind of moving towards the end, let's say people are interested in, let know, let's say they've read the p [01:20:00] and s paper or your, where does value come Paper? Where does value come from Paper and say, okay, this sounds very cool. Um, I wanna get started with this kind of stuff.

    So what would you, uh, what, what are some kind of resources people might use? I mean, there's obviously references in your papers, but those aren't necessarily the best. Places to start. 

    Keno Juechems: Yeah. 

    Benjamin James Kuper-Smith: Um, so I don't know, for kind of reinforcement learning, or not exactly what we would call the optimal utility paper in terms of research area, um, judgment, decision making, optimality kind of thing.

    But do you have any kind of suggestions where people might start and what to 

    Keno Juechems: Yeah. So for the, so starting with a more economically optimal utility functions paper, I did find it kind of hard to find good, good tutorials or resources on that. When I started with this, I mean, this was like at the beginning of my PhD, not, not necessarily for this [01:21:00] paper.

    I'm, I'm not. So a good resource is for sure the, the Kahneman and Treky Prospect Theory paper that's relatively straightforward. And to have this sort of described in, um. In words, like, not necessarily the math behind it, but to get a sense of why certain, like why economists have come up with all of this.

    I really enjoyed reading a book by Richard Thaer, who received the Nobel Prize in economics a while ago, uh, called Misbehaving, I think. So it's a collection of lots of different behavioral economics findings. And he is also kind of, 

    Benjamin James Kuper-Smith: I mean, that's his biography, right? Autobiography. 

    Keno Juechems: Well, it's a scientific biography maybe, but 

    Benjamin James Kuper-Smith: Yeah, yeah, yeah, yeah, exactly.

    It's not about, it's like family life. Yeah. 

    Keno Juechems: Yeah. So I, I really enjoyed reading that and it did give me some, 

    Benjamin James Kuper-Smith: yeah, 

    Keno Juechems: some valuable background knowledge about, uh, my research. But yeah, you probably won't get [01:22:00] around reading some fairly tedious papers along the way, I guess. So that's like the, maybe slightly.

    More difficult area, but for reinforcement learning, uh, because it's such a big thing at the moment. There are a lot of tutorials and online resources that, um, some of which are really very good. So the thing that I really enjoyed and, uh, where I sort of learned about the, the more machine learning side of it was David Silver's, UCL Reinforc Learning Course.

    So that's fairly corporate. 

    Benjamin James Kuper-Smith: The ones on YouTube, 

    Keno Juechems: right? That one's on YouTube, yeah. 

    Benjamin James Kuper-Smith: Started that. Never.

    Keno Juechems: Yeah. Um, 

    Benjamin James Kuper-Smith: yeah, I enjoyed the first two lectures actually. I never listened, watched the rest. 

    Keno Juechems: Well, uh, it, it is good if you have a reason to watch it. I think like it's, yeah, you sort of need the motivation to, to be able to use it fairly soon afterwards, I think. [01:23:00] Um, yeah, there's that, um. 

    Benjamin James Kuper-Smith: Is the, the, the reinforcement, the reinforcement learning book, a good starting point, or I've heard from some people that it gets basically what you're interested in as a psychologist and neuroscientist is like the first few chapters and then it's just computer science stuff that no one, like we wouldn't care about.

    I, I haven't read it yet. 

    Keno Juechems: That's right. So the first few chapters are definitely the one, the ones that you'd be interested in as a psychologist and afterwards, depending on what your specific area of of interest is, it will probably interesting probably, but it becomes more, more technical towards maybe after chapter, chapter seven or eight.

    But it, it is a, it is well written and I, and I enjoyed it as well. And so Chris actually, uh, has a, a sort of. Manuscript that he uses to teach a course, I think it's called How to Build a Brain. So that's, uh, this is, this is like aimed at undergraduates in their [01:24:00] third year. I think. 

    Benjamin James Kuper-Smith: That sounds like my level.

    Keno Juechems: And, uh, yeah, it's, uh, I haven't read all of it, but the bits that I read, I quite enjoyed it and it's quite accessible, so that, that's maybe a good, easy starting point for people. It's on, it's on the, on the lab website somewhere. 

    Benjamin James Kuper-Smith: Uh, okay, good. Then I'll find it. Yeah. How to build a brain from scratch. 

    Keno Juechems: Yeah, that's, uh, yeah, 

    Benjamin James Kuper-Smith: that sounds like it.

    Oh God, it's 14 megabytes. There's lots of pictures. 

    Keno Juechems: It's, it's, 

    Benjamin James Kuper-Smith: oh, oh, it's actually like a book kind of thing, or it's 211 pages. Oh, there's, that's, that's pictures. Yeah. 

    Keno Juechems: Yeah, 

    Benjamin James Kuper-Smith: yeah. For a second. I was like, oh, that's, that's a long thing. But yeah. Uh, that's cool. I didn't. Probably learn more than, than I'm willing to admit on that one.

    Although I guess that's, 

    Keno Juechems: well, I certainly learned a lot from it too, 

    Benjamin James Kuper-Smith: by the way. One thing that just I also realized is that the, um, about like resources, one book that I've really enjoyed, I'm not, I'm not even sure how much it's gonna help you become a better scientist, but I find it, I really love reading [01:25:00] it.

    Is the, uh, the ka ky kind of double biography or working relationship biography by Michael Lewis? Mm-hmm. Um, I dunno whether you've read that. Yeah, I don't think 

    Keno Juechems: so. 

    Benjamin James Kuper-Smith: Yeah, that's a really, I mean, Michael Lewis is the, if you dunno him, he's a nonfiction writer who, uh, for, he had like a few years where like all of his books were made into cinema film, like, into films, uh, like, uh, the Big Short Moneyball, the Blindside, a lot of those.

    I'd really be surprised if the Ka ky book isn't made into film fairly soon because it's just, it's just absolutely lovely. Mm-hmm. Uh, so that might be another one. 

    Keno Juechems: No, that sound great? Yeah. 

    Benjamin James Kuper-Smith: Okay, cool. Then maybe as my kind of last question is, uh, what's next for Ken Union and maybe related to that, just something I meant I realized or noticed something, I noticed something you said earlier about, um, you know, your bachelor's was very clinical and you weren't really interested in that [01:26:00] on your kind of St John's website thing.

    It does mention though you are interested in applying this to patients with mood disorder. Is that kind of the outlook for the next few years or, 

    Keno Juechems: uh, so that, that would be some aspect of what I'll be working on probably for the rest of my fellowship in some ways. 

    Benjamin James Kuper-Smith: How long is that? Uh, 

    Keno Juechems: so I have fixed a year and a half and probably an a year extension afterwards.

    Um, so still, still a good amount of time, but it's now, so if we really wanted to do work with clinical, uh, populations and it's already maybe a bit late to finish it in the timeframe, um, but it's certainly something I'm. I'm interested in to collaborate with people. I'm not seeing myself move into that, like, as a sort of full-time sort of commitment.

    I'm, I, I do get most enjoyment out of the, the basics of neuroscience and psychology and, uh, yeah. So more practically, like, uh, I still have a bit [01:27:00] of time on, on this fellowship and, um, partly because of the pandemic, I've switched a bit more to some theory work with kind of machine learning methods to, that would generate kind of what I talked about earlier that, so that, that might.

    Give you a teaching signal that's like, oh, you might want to explore this doorway, or you might want to use the information you have from previous, uh, previous tasks to sort of come up with good guesses of what you might be doing in a certain kind of task. So I'm working on, on that fairly technical side of things at the moment, partly because I didn't have the chance to do more brain imaging, which I would've normally done, I guess.

    Benjamin James Kuper-Smith: Mm-hmm. 

    Keno Juechems: Yeah. And then we'll see what happens there. 

    Benjamin James Kuper-Smith: So after that, it's just open geographically. Are you, you fit in Oxford now for a while? Do, do you wanna stay there or is, I dunno, I was just curious in terms of I haven't been in one place for longer than [01:28:00] 18 months, basically. 

    Keno Juechems: Mm-hmm. 

    Benjamin James Kuper-Smith: Uh, since I left school, I think.

    Um, so I'm always fascinated by people who spend lots of time in one place. Um, 

    Keno Juechems: um, yeah, I've, I really like. Oxford, um, as a place and also as a research environment. And there are a lot of people who work on similar problems, not necessarily directly in Oxford, but London is just around the corner. So it's a great place to be, but it's also obviously very difficult to stay after, after a while.

    Um, so, um, 

    Benjamin James Kuper-Smith: yeah, 

    Keno Juechems: uh, I'm not like fixed on the idea that I would, that I'm going to stay here realistically as well. Um, and then, yeah, I would favor somewhere in Europe, but I'm not necessarily fixed on it. 

    Benjamin James Kuper-Smith: Okay. I guess we'll.