11. Jesse Geerts: Finding a good PhD project, reinforcement learning & cognitive maps, and deciding when a paper is ready

Jesse Geerts is a PhD student at the Sainsbury Wellcome Centre at UCL, in the lab of Neil Burgess. We met a few years ago when we were in the same cohort of the Dual Masters in Brain and Mind Sciences, hosted in the first year in London by UCL and in the second year in Paris by UPMC and ENS.

In this conversation, we talk about Jesse's new paper in PNAS, what it's like to do his PhD programme, how to know when a paper is ready to be submitted, and a bunch of other topics.

BJKS Podcast is a podcast about neuroscience, psychology, and anything vaguely related, hosted by Benjamin James Kuper-Smith. New conversations every other Friday. You can find the podcast on all podcasting platforms (e.g., Spotify, Apple/Google Podcasts, etc.).

Timestamps
0:00:05: During the recording, there was a 4-second delay, but I hope I edited it out alright
0:01:16: Finishing our PhDs
0:15:23: Jesse's experience in the Sainsbury Wellcome PhD Programme
0:23:41: Deciding what PhD project to do (and with whom)
0:54:15: Ask for help (unless the solution can be googled)
0:58:30: Discussion Jesse's PNAS paper
1:30:22: Idea for a new podcast: Ben's Roast
1:33:45: Evaluating whether a model works
1:39:21: When is a paper ready?
1:47:00: What's next for Jesse P. Geerts?

Podcast links

Jesse's links

Ben's links


References
Geerts, J. P., Chersi, F., Stachenfeld, K. L., & Burgess, N. (2020). A general model of hippocampal and dorsal striatal learning and decision making. Proceedings of the National Academy of Sciences.
Geerts, J. P., Stachenfeld, K. L., & Burgess, N. (2019). Probabilistic successor representations with Kalman temporal differences. arXiv.
Kuper-Smith, B. J., Doppelhofer, L. M., Oganian, Y., Rosenblau, G., & Korn, C. (2020). Optimistic beliefs about the personal impact of COVID-19. PsyArXiv.
Stachenfeld, K. L., Botvinick, M. M., & Gershman, S. J. (2017). The hippocampus as a predictive map. Nature Neuroscience.

  • [This is an automated transcript with many errors]

    Benjamin James Kuper-Smith: [00:00:00] Yeah, that seems to be like a two second delay or something, but I guess it's just what it is. Again, when I edit it, I can take out these, um, gaps, but it's fine. It does feel a bit like you're sending, like I've sending out a signal and then once I've start, once I've stopped speaking, like I wait for like two seconds to see your reaction.

    Jesse Geerts: Uh, no. That's terrible. Yeah, I, I have a lot of, uh, video calls, um, from here and usually that goes fine on Zoom, but I think Zoom, um, handles like bad bandwidth quite well and still gets over the message in like a sort of like smooth 

    Benjamin James Kuper-Smith: Okay. 

    Jesse Geerts: Uh, transition. Um, whenever it's like something else like, I don't know, um, FaceTime or Google Meet or something, it's much worse.

    Benjamin James Kuper-Smith: Hmm. Okay. Uh, but just quickly, is there a, is the network even really bad when you are like right next to the router? 

    Jesse Geerts: I am right [00:01:00] now, right next to the router. 

    Benjamin James Kuper-Smith: Yeah. I'm just sitting on the router 

    Jesse Geerts: pretty much. Okay, 

    Benjamin James Kuper-Smith: good. Maybe, maybe that's the problem. You are blocking the wifi with your butt 

    Jesse Geerts: too heavy for the wifi 

    Benjamin James Kuper-Smith: anyway.

    Have you, have you been, uh, doing, or how are you, um, should, shouldn't you be finished at some point soon? 

    Jesse Geerts: Yeah. Um, yeah, good point. I, I, uh, should have originally been finished last October, which would've been the four year mark. Um, but partly 'cause of COVID and partly just because I wanted to finish, um, another project that I've been working on before the end of my PhD, I got a six month extension.

    So, um, that runs out the end of March. So I'm aiming to, uh, finish then. Um, so I'm, yeah, recently I've been trying to concentrate on writing [00:02:00] up a, uh, paper, um, which constitutes the second project of my PhD. Um, and, uh, so that paper, um, combined with the, the one that's out now should, uh, should form like based my thesis.

    Benjamin James Kuper-Smith: Mm-hmm. Actually, I have one question just about, uh, the, the format of, or not the format, but one question about, so you have this Preprint, right, which I think was a conference presentation about the Kaman filter, uh, Kaman temporal differences. 

    Jesse Geerts: Yeah. So I, I'm not counting that one. 

    Benjamin James Kuper-Smith: Okay. Yeah. Because in computer science they often I heard, or at least in parts of computer science, they, I remember one guy saying like, well, we don't really do papers, we just do conference, like papers, like, uh.

    You know, you get that exception that, that then that is your presentation. I was kind of curious whether, but that's, that's, yeah. Well, you're not a computer scientist, so that doesn't count then? 

    Jesse Geerts: No. Well, I mean, I, I, I [00:03:00] dunno what counts. So in the UK you don't, um, you don't officially need a published paper to finish your PhD.

    Uh, there's no like, strict requirement for that. These conference papers, I have two actually. Um, they, they sort of count as a paper, but they sort of don't. Um, the, the nice thing is I think I, I've seen that a lot in computational neuroscience where maybe they, they have some kind of small conference paper, uh, or maybe for a workshop or, yeah.

    Or, or a conference like, uh, uh, CCN. And then they work that small conference paper out into a, an actual journal paper, which is what I'm trying to do for, uh, the second project. 

    Benjamin James Kuper-Smith: Uh, so, so this is basically the second project then? 

    Jesse Geerts: Yeah. That common temporal differences thing is, uh. Uh, it's like the basis for my second project.

    Yeah. 

    Benjamin James Kuper-Smith: And the extension is from within the PhD program, or is that via Neil or 

    Jesse Geerts: No? Uh, from within the, within the PhD program, they, uh, they allow us all to take six [00:04:00] months extra. 

    Benjamin James Kuper-Smith: So it's not just because you were a good boy and ready publish something, it's just to anyone who gets accepted into the, into the program.

    Jesse Geerts: Um, so I think, so originally when I joined, they actually told us that we all could get a whole year extra and, um, kind of backtracked on that a little bit. So I, I, I counted on that for a while. Um, and then like, so in my last year, I told some of the people at the building like, oh yeah, I might take that extra year because I'm, I'm nowhere near finished.

    Uh, and then I heard that actually you can't get a whole year extra. Um, but if you have a good reason, you can get a healthier extra. Um, and I think I, I, I wouldn't have wanted to take more than a he year anyway. So, uh, half year is okay now though, so I was gonna take this extension anyway even before the pandemic.

    Now with, with COVID, it's, it's actually quite hard. I, I think a lot of people are having this right now, but it's actually quite hard to keep concentrating and keep on going at it. Um, so it's gotta be tight, even for [00:05:00]that healthier extension. I'm gonna have to work quite hard in this, these last months and somehow turn a switch, I think.

    Benjamin James Kuper-Smith: Yeah, I mean, but that's, that's kind of, uh, that's, that's a good move to say. Everyone can have a year extra when you ask it to say, wow, that was just a joke. Didn't really mean that. 

    Jesse Geerts: Yeah, it's, it, it, it's a bit chaotic because it's a very new PC program. I am actually on the very first cohort and during that first year, they were still setting up the program.

    A lot of things weren't quite clear yet. The person that said, um. That we could have a whole year extra actually doesn't work at the building anymore. So they were still really, they were still really figuring stuff out. Um, so I, uh, I think we should be a bit forgiving of, of the chaos. 

    Benjamin James Kuper-Smith: Yeah. And getting four and a half years is also more than you get in pretty much any other program, right?

    I mean, often it's three years and you're out. Good luck to 

    Jesse Geerts: Yeah, especially in the uk I think, I think in, uh, continental Europe and, um, and the United States, it's normal to [00:06:00] take a bit longer. 

    Benjamin James Kuper-Smith: Yeah, that's, yeah, I mean it really depends on the, on the format. I mean, when I started in a Berlin, it was very much a clear case of like, you have three years, you have to like publish something.

    Uh, and if you do that, if you hand in within three years, you can get a one year postdoc. But, uh, it was very clear, like if you are, you know, if you're not finished within three years, then either your supervisor pays you or. You are unemployed. I mean, that's all, you get a different position, right? But that was very strict there.

    But yeah, you're right. I mean, I mean, that's what I like about the thing I'm doing now is that I've, you know, it's set up for four year. I have a salary for four years. 

    Jesse Geerts: Yeah, that's great. 

    Benjamin James Kuper-Smith: And that's so much nicer than like being kicked out after three years. But actually, uh, I'm curious. So this is something that I've changed my mind on during my PhD a bit.

    Um, so I'm curious what, how, what you think about that. Um, so when I was doing my like bachelor's and masters, I thought, you know, I really want to [00:07:00] finish this quick and I want to, um, you know, a PhD can take three years, so I'll do it in three years and then I'll do a postdoc and all that kinda stuff. But like now that I've, that I'm actually doing my PhD, I'm realizing like I'm actually kind of enjoying it and I, or not kind of, I'm really enjoying it.

    And, you know, I don't wanna take forever to finish my PhD, but right now I really had, I have no problem taking like four or five years because I feel like it just, you know, I can just do research and I don't have to, you know, do much admin or teaching or anything. I can basically just, it feels like being paid to just do whatever I want to do.

    So in a way I feel like I, you know, if I could do this for five years, I probably would, um, if I, if I get money for it, I was So, I'm just curious whether you, whether you changed your mind on that or how you thought about that. 

    Jesse Geerts: Yeah, definitely. I was, I, I think I went through a similar process. I think from a pure strategy point of view, um, if you wanna build your career in academia or something, the best thing to [00:08:00] do is probably to stick around longer bit in your PhD because probably when you're a PhD student, that's the first time you're really doing proper research.

    Uh, and often the first time you're doing research in your particular field and it's gonna take you like one, probably two years to even be. Call yourself somewhat of an expert in that field. Um, and I think a lot of, a, a lot of people that I know, including myself, you, you, you feel like you've just started getting going in the third year or something like that.

    So taking one more year at the end of your PhD might be very fruitful in terms of actually producing some publishable research. Um, uh, whereas jumping to the next, um, job where indeed you also have to do teaching, uh, and maybe other things, um, it might take a bit longer before you get something out. 

    Benjamin James Kuper-Smith: I mean, what I also always thought is that, um, when you look at, at least, I think probably actually most of the postdoc funding, the, you know, it's, it's often [00:09:00] restricted to certain amounts of years after your PhD or something like that.

    So there might be some funding schemes that say, you know, you can apply to this within five years of finishing your PhD. So to me it always seems like if you just take one more year and like, get out a really cool paper or just a decent paper, right. Just having that more. I, I dunno whether anyone's really gonna look too closely in terms of how long it actually took you to finish the PhD.

    I think it's more about like, what can you do? And it seems to me being able to, yeah, just taking a year to show that you can do something cool is way better than, you know, finishing six months early or whatever. 

    Jesse Geerts: Yeah, absolutely. I think, um, if you have the luxury to do so, if there's, if there's some money, I would recommend that.

    Benjamin James Kuper-Smith: Yeah, exactly. Yeah. It's, it is a, a luxury. Yeah. But I'm in the weird position that I, so, you know, I had, as I said, I have a pretty much guaranteed four year salary. For my PhD to, to finish my PhD. Um, but then when the COVID pandemic happened, [00:10:00] we very early on started doing this project related to COVID and risk perception, um, which roughly relates to some stuff that my supervisor did before, and we kind of just jumped into it hoping that maybe we can like help or do something important, like, sorry, important per se, but like, you know, do something to inform the situation and kind of use our expertise, um, specifically his, in terms of content.

    More on mines mine more in terms of just being able to run online studies very quickly, which, yeah, we tried to just use that for the good. And, uh, so I've been basically working on that now for, well, since the beginning of the pandemic. Well, there was someone who did something else, but I've been working on that for like half years.

    So somehow I'm ended up in the position where now where I practically speaking have basically a three year PhD. And two months or so. 

    Jesse Geerts: Um, but is this gonna be part of your PhD, the COVID research or, 

    Benjamin James Kuper-Smith: uh, no, no, it's completely unrelated. I mean, not completely unrelated. It's, you know, it's about, [00:11:00] uh, I don't know.

    I mean, very vaguely, it's psychology, so that's, it's related in that sense. Um, but no, I don't, um, I mean, many people, I, I've talked to people about this and they said, well, wait until you finish your PhD and see, maybe you have to make this part of your PhD, like if you don't get publications out or something.

    But right now I'm so optimistic that, uh, I won't. Uh, needed to make this part of my PhD, 

    Jesse Geerts: but that, that's a nice scheme though. Four years for your actual PhD. 'cause um, so the program I'm in is more of a three plus one, where, uh, the, in the first year you get to do some courses, you can rotate in some different labs, try out what you want, and then there's three years for extra research, which, yeah.

    Uh, in my experience, it like has been pretty short. Like first you think, oh, I, I have three years and after one and a half years you still need to do so much. It, it kind of like feel like it flew over. 

    Benjamin James Kuper-Smith: Yeah. I guess especially if you are, you know, as you mentioned, if you're doing something that's slightly different from what you did before, I mean, I guess technically you could say the [00:12:00] rotations are for that for, you know, you can do rotations, so you get started with one project, then you've got a few months, um, on that project, that on which you do your PhD.

    But I mean, even, so in my case, like. I even did a, like, what I'm doing isn't a million miles away from stuff I did before, but it still feels like I spent the first year almost just getting to know the field. Um, and, you know, for the first time, like having to just figure out it, it seems to me it's very, you read a paper very differently if you are a master student or a bachelor student than if you're a PhD student.

    Just because you have a completely different, like yeah. The, the purpose and the reason why you're reading is completely differently. So it feels like to me, I still had to like, yeah, I just spent the first year figuring out what the field is, uh, and I'm, I'm not finished with that yet. 

    Jesse Geerts: Yeah, absolutely. I, I, I don't know about you, but I, I feel like I've, I read fewer papers than I used to, but, but more thoroughly probably, 

    Benjamin James Kuper-Smith: yeah.

    It, yeah, I think for me it really comes and goes [00:13:00] in, um, what's the word? Uh, you know, sometimes I'll have a few months where I'm basically just. Running my experiment, analyzing the data and reading fairly little. But then I sometimes have weeks where I, you know, reading full time and scanning lots of papers and reading a few papers fully.

    So I dunno, on average it's probably fairly similar, but it's, uh, it's not as consistent as it was before. 

    Jesse Geerts: Yeah. And I guess my, my focus has, um, changed from mostly the sort of introduction and discussion when I was a undergrad or master's student to mostly the methods and results sections. Uh, or, or, or just like a quick scan of the figures or whatever.

    Uh, later on in my PhD. 

    Benjamin James Kuper-Smith: Yeah, definitely. Although I had one really weird thing. So when I was, um, when we were doing our masters, I was living with someone who was doing a PhD in some biomedical engineering kind of [00:14:00] imaging thing. And she actually said the opposite, which really confused me at the time. And I thought, well, maybe I'm, you know, she's a PhD student.

    I'm not, maybe I got this wrong because she said, yeah, since I'm, since I'm a PhD student, I mainly read introduction and discussion. And then I had this whole point about like, no, you have to read the, the methods and the results. Like, that's right. Like if you do research, you have to do that part. But she argued against it.

    But somehow, yeah, I wonder, I mean, I think she, she got a good, I mean, she got a PhD from UCO, I think she must have done something. All right. But that still confuses me how she only, 

    Jesse Geerts: she's an ideas woman. 

    Benjamin James Kuper-Smith: Yeah. I don't know. I mean, yeah, but it wasn't engineering, like, engineering sounds like less ideas and more execution.

    Um, so I don't know. But I, I, I agree with you, but I'm still confused with that one discussion I had with her. 

    Jesse Geerts: I mean, there are definitely still papers that I read in that way as well, or like, maybe just like the abstract and discussion or whatever. Um. It depends on the level of detail at which I would want to understand the paper, I guess.

    Benjamin James Kuper-Smith: Yeah. For example, uh, I mean the, the [00:15:00] paper of yours that we're gonna talk about later, I haven't read all the small print in the back about exactly how you did the analysis and the simulation. Maybe these kind of papers that are still like vaguely related, but more on the, I mean, not directly related to what I'm doing right now.

    Yeah. You, I don't go into all the details. 

    Jesse Geerts: Yeah. Same for me. 

    Benjamin James Kuper-Smith: Yeah. But, um, then just quickly, or not quickly, um, take as long as you want and Chris, about your PhD program because, uh, it sounds pretty cool from the outside. Um, could you, uh, so what maybe could you, how should I say? Yeah, just kind of give a brief outline of what it was like to do the program.

    I mean, you already mentioned that you have these rotations at the beginning. Um. I dunno. Let's say, let's say maybe there's someone who's maybe someone's listening is interested about applying for the program. 

    Jesse Geerts: Yeah, absolutely. Um, yeah, I can tell you a bit what, what the first year was like and how I like ended up in my, my [00:16:00] PhD research afterwards.

    So yeah, for those listening, I'm at the Bury Welcome Center, um, for Neuro Circuits and Behavior is the full title. And it's a program that was founded now, like four years ago. I was on the first cohort. Um. Really sort of the idea is to marry a theoretical neuroscience institute, uh, which would, which already existed, the, um, Gatsby unit and a systems neuroscience, um, institute, which is brand new called the Saint three Welcome Center in one building with, uh, its own PhD program as well, uh, where you get, um, a first year of some training in experimental neuroscience techniques and some courses on theoretical neuroscience, um, after which you, uh, ideally join one of the labs that is hosted in the building.

    And yeah, the original idea of this program at the outset was that in, I [00:17:00] guess like sort of the, the field of neuroscience, there's quite a, a, a large disparity to, um, between working in people, working in theory and people working in experiments. Um, and I, I think the, the goal of this program is to, uh, to, uh.

    Make that gap a bit, a bit less white now, when I joined the building had really just been built, um, and the program had just been conceived. So, um, the, uh, yeah, this, this has changed a lot over the years. But when I, so when I joined, it wasn't much of a program yet. Our, um, courses consisted of some, uh, some classes by the PIs that were in the building.

    There weren't that many yet. And walking along with the already established guest, so 

    Benjamin James Kuper-Smith: not many means there was like five or something, right? 

    Jesse Geerts: Yeah, yeah. So at that moment, I think there was, um, uh, John O'Keefe, the, uh, the Nobel Prize [00:18:00] winner who discovered play sales in hippocampus, um, cha Branco. He does, uh, research about innate behaviors.

    Then there was Troy Margay and Adam Camp were already there soon after that. Also Tom Melo, uh, joined and Sonya Hofer. Um, but that was when I was already in the building for, for half a year or so. Yeah. And the Complete Gatsby unit, uh, Gatsby Computational Neuroscience Unit, they, uh, were already in the building.

    So our, the, the, the non-existent PhD program that I joined consisted of going into a lot of those, uh, already, uh, existing courses, uh, that were given by the ga uh, Gatsby unit. And, um, a lot of courses given by Adam Kth, uh, who is an absolutely great guy, uh, explaining us, um, a lot of techniques that are used in, uh, experimental neuroscience.

    So this consisted of [00:19:00] learning the basics of electrophysiology, uh, calcium imaging, how does a two photo microscope work and so forth. Uh, which was really cool. Um, albeit a bit, 

    Benjamin James Kuper-Smith: have you had any prior experience? Or training in these, like, experimental systems, neuroscience techniques, or was that new to you?

    'cause if I remember correctly, your math projects were not on those topics, right? 

    Jesse Geerts: Yeah. Um, definitely. Um, so all of this, um, all of this experimental neuroscience sort of Yeah. So the building specializes in rodent, um, systems, neuroscience. So in a building, most research, um, uh, happens with rats and mice. Um, and all my prior experience, uh, was more in human, EEG, uh, human, FMRI, et cetera.

    Um, so this was all very new for me. [00:20:00] And the, the, the, the sort of, yeah, it, it, it's, it's pretty interesting the the level at which. A researcher in rodent neuroscience is, uh, rodent neuroscience, uh, is tinkering with the, um, measuring apparatus itself is really very different, uh, between, um, between, for example, fmri and, um, and rodent phys.

    In, in, in FMRI, there's usually, um, a radiologist, uh, who works there who wants or understands the machine and who helps you, uh, as, um, as a researcher to, to operate that. Whereas rodents, electrophysiology, labs, they, they make everything themselves. They, they, they twist the teros themselves. And, um, yeah, there's, there's a whole sort of very artisan style style about it where everybody does it in their, in their own way.

    Benjamin James Kuper-Smith: Uh, it sounds pretty, it's, yeah, it's, it's weird how it's, even though the goals are [00:21:00] often the same, how, it's such a different world, you know? I mean, yeah. I mean, as, as you mentioned, like if you have FMRI, you. Ideally understand the thing, but you won't really be tinkering around with the FMI machine. I think most people who are in charge of an MRI don't want you to really do anything with it.

    Jesse Geerts: Yeah. There's a lot more DI ying going on in a, uh, rodent neuroscience lab. 

    Benjamin James Kuper-Smith: Yeah. Is it, is this a matter of cost or like how much does it cost to build a tero, because again, MRIs cost millions, so it makes sense that you don't want someone without an experience just tin around, but somehow I imagine that these animals things would also cost a fair amount of money.

    Right. 

    Jesse Geerts: So, um, teros cost almost nothing. Um, and a two photo microscope will cost maybe a couple thousand. Uh, but that does nothing compared to a, um. Compared to an fmri uh, machine, which I, I'm not sure what the cost is, but I would estimate it's more like a million pounds. It's more comparable to e eeg.

    Yeah. [00:22:00] 

    Benjamin James Kuper-Smith: Yeah. And with fmri, of course, the, if you're in a big city, the problem often becomes the place to put it in. Right? The room. You need like a specialized room for it, that has to be large enough. Uh, which, you know, if you're at UCL in the middle of London, that that in itself can become a problem. 

    Jesse Geerts: Yeah, absolutely.

    Yeah. So that is all changing a little bit. So I guess the, especially the phys community has lived with these like, um, homemade teros for decades, but more recently these labs are moving to towards like CMOs scanning probes with maybe a thousand contacts, um, in the brain, which no, no lab can make themselves.

    So these are industry made. 

    Benjamin James Kuper-Smith: Is that, sorry, how does that relate to, so how does that relate to the neurop pixel stuff? So I've heard of that. 

    Jesse Geerts: Oh, that that is, that's what I'm referring to. Yeah. A neurop pixel. 

    Benjamin James Kuper-Smith: Oh, that, is that, okay. Yeah. Okay, cool. Uh, so, okay, so you have as a [00:23:00] company who, that builds those and sells 'em to labs then?

    Yeah. 

    Jesse Geerts: Yeah, exactly. Um, I think there's a, um, there was a European Union grant, but I, I haven't exactly followed that, but, um, I think maybe neuron makes them, don't quote me up. Maybe, maybe cut that bit out. I, I, I'm not, I'm not entirely sure who makes the neuron. Okay. 

    Benjamin James Kuper-Smith: Okay. Okay. Yeah. Um, but you, I mean, we should also say that you don't actually work with these things, right?

    I mean, you were trained in them in the first year, but from what I can see, you're not actually doing any electrophysiology anyway, right? 

    Jesse Geerts: Yeah, that's true. So at the start of my PhD, um, I. Didn't quite know what I wanted to do. I knew a couple of things that I were, was interested in. Uh, one of them is like motor neuroscience and optimal control.

    Uh, another one was reinforcement learning, um, but also perception. So it is really like all [00:24:00] over the field of neuroscience. So that's in terms of the subject of study. Then in terms of the methods, uh, I found rodent neuroscience very appealing just because of the actual precision that you can record actual neurons with.

    And of course, um, that gets very close to, um, how the brain actually operates. There's an alarm going off. This is very annoying.

    Um, so yeah, so that, that gets very close to sort of the units of actual communication in the brain, especially now with the, the advent of neurop probes and the like, uh, which. Makes us able with to actually record all of those neuros at the same time. But when I started trying the actual lab research, I found out two things.

    One is that, um, I'm not particularly good at, uh, very Dexter lab work. Uh, and two, uh, probably more importantly because I, I'd probably [00:25:00] learned that, but I, I don't enjoy it. Um, I did not enjoy working with the animals, and I did not particularly enjoy the prospect of a, a lot of my PhD being about this kind of tinkering, uh, denoising, the setup, et cetera.

    Benjamin James Kuper-Smith: Yeah. Okay. Like the tinkering, the, the figuring out exactly how to get the setup to work. Um, I mean, with the animals, was it, what did you not like? Did you not like the whole surgery part or just the killing animals part, or, uh, what exactly did you, or was it more about the whole set experimental setup thing?

    Jesse Geerts: Yeah. Well, I think I, I, I think like, just to be clear, nobody really enjoys getting the animals. Um, 

    Benjamin James Kuper-Smith: well, 

    Jesse Geerts: um, and I'm not, I, I, I'm, I'm also not against using, um, animals research in any way. I, I use a lot of the data that, uh, comes from, uh, experimental labs. The, our own lab [00:26:00] does a lot of, um, run research. I just thought, uh, I'm probably not the person that should be doing it.

    What I feel more comfortable with is, uh, spending a lot more time. Uh, on thinking about the, the research questions are doing analysis and this kind of thing, which I was used to for more, uh, human cognitive neuroscience before. Um, the sort of, yeah, the, the amount of time spent on, on actually gathering the data itself, uh, during which you're just, you've just accepted, oh, this is, this is the question that I'm gonna answer.

    Like, you now I, I need to gather this data, this data as well as I can with as little noise as possible. That, that, uh, process is something that, um, I see some colleagues of mine, for example, really enjoy. And, and, and, and, and if you're good at that, that's, that's like a really, a really nice thing. It's not something that I myself enjoy.

    Benjamin James Kuper-Smith: Yeah, [00:27:00] I, I get that completely. Um, I think that's also maybe, I mean, I never did any of this electrophysiology and animal stuff, but you know, when, you know, when we did our masters, especially the year at UCO, I think I, I, I fell very much in love with the whole literature and the stuff, you know, stuff you can find out with it and the stuff that they did find out with it.

    Um, so I really considered at least doing some sort of project with this kind of electrophysiology. But I think in hindsight, I would've run up the same problems as you. I don't think I would've had the. The patience almost just to, to set up something like so specific for so long. I think I'm, I, I enjoy thinking about like the bigger questions so much that probably good that I didn't get into anything like this.

    Jesse Geerts: Yeah. I, I felt pretty lucky that, um, my program's pretty free in what you wanna do. Um, so when I told, so the, the rotation in which I tried the, um, the animal work was with, uh, Adam Camp when, um, I [00:28:00] told him that the, like, this is not for me. I, I, I love the lab, I love the people and, and the questions are interesting, but this, this is just not for me.

    He was very encouraging and, uh. Said like, oh, well maybe do another few rotations and, and see what you like. 

    Benjamin James Kuper-Smith: This actually relates to something I wanted to ask anyway, which is, yeah, I mean, the general question is maybe like, who do you think this program is for? Or like what kind of questions should you, let's say you think you want to do this, I mean, not this program, but like this kind of program doing electrophysiology or something like that.

    Um, like what kind of questions should you ask yourself to figure out whether this is the right thing for you? 

    Jesse Geerts: Um, yeah, that's a great question. So I think if I, if I look around me at the other SWC PhD students, um, there's really quite a wide array of people. So. I, I, I, I don't think you need to be a rodent, electric physiologist or a, or a rodent neuroscientist to apply.

    [00:29:00] Um, there's a, a, a, people with a whole, um, range of backgrounds have been hired in the past. People who are just engineers or that move wanna move into neuroscience. There's some psychologists. Um, and there is, um, there are also, um, there, there's a guy in the year below me who actually started, um, a rodent electro FIS project and did that for about one and a half years.

    Then decided, this is not quite for me, because I, I, in his own words, I'm too, I'm too lazy for the, um, for, for the rodent experiments. So he moved to a theory project, and that is fine with the supervisor as well. I. So, yeah, I, I think sort of the, the, the, the answer to your question lies a little bit in the, in the application, uh, process already where they just asked us to write an, a short essay, 500 word essay or something on what we found the most interesting unanswered question in neuroscience.[00:30:00] 

    Benjamin James Kuper-Smith: What did you, did I have to read the essay, but like, what did you 

    Jesse Geerts: Um, uh, so I actually, uh, wrote about, um, sensory neuroscience at that moment and, uh, this, um, sort of predictive coding hypothesis, um, or sort of hierarchical predictive coding idea. The idea that instead of, instead of sort of vision being a bottom up, bottom up thing, uh, the visual system has a sort of up a, a top down generative model of what is, uh, gonna, what is gonna appear on the retina next, uh, which is not at all, uh, what I moved into for my PhD, but, uh, yeah, uh, yeah, that, so, so maybe that's another, uh, thing that I have learned during my PhD is that.

    There are a lot of very interesting questions in neuroscience and as I, I already refer to sort of, uh, the reward learning field that I actually went into, but also motor neuroscience and perception and I, I think a lot of neuroscientists will find that they, they find all of those fields quite interesting.

    [00:31:00] Um, and what you should choose for your PhD PRO project. Therefore, unless you're really tied to this one particular question already, but, um, I, I think. Choosing a PhD project should also involve trying to think a of a team that you really like working with a supervisor that you, uh, that you like, and, uh, just creating this environment for yourself where you can thrive.

    Benjamin James Kuper-Smith: Mm-hmm. Yeah. I, what I find really weird about choosing a topic is that I find it really hard to judge from the outside how much you actually will enjoy working inside the topic. I mean, so I had this situation right where I started the PhD in Berlin and then quit that after about half a year because I, I just didn't, yeah, I just didn't really like the, um, I didn't really believe in what I was studying and I wasn't entirely sure that what I was studying wasn't like an artifact and like an experimental artifact or something.

    Jesse Geerts: So it was, [00:32:00] was it for you more the topic than the environment that make you, um, decide to stop that? 

    Benjamin James Kuper-Smith: Yeah, I mean, so the, I mean, it was a, a complicated thing. I think there were a lot of things going in to the decision to quit that. Yeah, it was, it was kind of weird that I think a lot of things that I expected to be a certain way were all a different way.

    And in hindsight now I realize some of them were probably good that they were that way. But yeah, it was probably like, at least, yeah, I would say it's probably at least like 70, 80% of that decision was purely based on the topic. Um, I mean, there were other things like the, I didn't, the, the methods were cool.

    We were using, so we were using like direct quarter to stimulation in, um, people who are having brain tumors removed. And it's a super cool technique with which you can, you know, one of the few techniques that you can actually use to test causality in the brain, in humans. Um, 

    Jesse Geerts: yeah, that sounds like a pretty unique, uh, set [00:33:00] of subjects that you have.

    Benjamin James Kuper-Smith: Yeah, it, that was, I mean, that was like half of the thing I was doing, but I also realized like, so the thing with this direct cord disc stimulation is that, you know, it was, I mean, it's super cool, just like I, I was, I spent a few weeks in the, uh, hospitals and like look watching them do operations and, and with people who were awake during the surgery and everything.

    Like, it's super interesting to look at, but I realized like I can't, um, how should we say, most of the work with these kind of thing involve just waiting around for them to actually do the operation. So, you know, you kind of have to hope that you get the right patient, which wasn't really that much of a problem.

    But you, so for example, you, so I never actually, so I mean, I quit before I actually decided, before we actually started collecting data. I was just at the phase where I was just checking out what they were doing, other experiments they were running. Um. Sometimes you'd have, you know, you'd, you'd have a patient and surgeries just take ages to do, and you have [00:34:00] to, you know, you basically just have to stand around there.

    And, um, so it's just, it's just a lengthy process of let's say like you have to be there for at least like two hours or something. And it might be that. So with these awake operations, they actually, they put them under full anesthesia when they open the skull. Um, but then, and when they remove most of the actual tumor, but then when they get to the edges of the tumor, then they wake up the patient to check whether, you know, they stimulate the surface area to see basically whether that impedes functioning.

    Um, because the thing is, you don't actually, when you look at a, a brain tumor, you can't really tell by looking at it what it is. It really looks exactly like the brain, at least in the cases that I saw. Um, it's, it's surprising. You'd think it looks like, you know, some like dark mass or something, but it looks exactly like the brain.

    So you need to stimulate the edges to see exactly what you're doing. Anyway, the um, 

    Jesse Geerts: so is, wait, is the reason that you, um, have to [00:35:00] wait there with the surgeon is the reason, um, because you wanna do the task with them afterwards? 

    Benjamin James Kuper-Smith: Yeah, exactly. So the idea is then that you, um, so once you've kind of taken up most of the tumor and you are there to the stage where you can start stimulating, uh, the brain, the idea is then that, so they wake up the patients and then you do some sort of task with them.

    And then that, for me would be the experiment I would've been doing with the patients. Um, and so basically, I mean, yeah, the reason you just have to wait around is so you don't miss the crucial part of when you do the task with 'em. Um, you don't have to obviously be there for the entire operation of where they like, you know, shave the head or parts of the head and all that kind of stuff.

    But, um, you kind of have to be there for, yeah, it's, it's gonna be at least two hours anyway, the, so I saw some of the operations where, you know, you'd go in there, you'd wait for a while, and then the patient, when they woke them up. They'd just be very groggy and very tired and [00:36:00] could, couldn't really function well because even though you say you wake them up, that doesn't mean that like full cognitive functioning, they're still kind of, you know, they, they've just been, I mean, I dunno whether you've been under general anesthesia, they've just been taken out of general anesthesia, so that's the state they're in and you get like 10 minutes or something in that stage.

    So basically, I mean, what I'm trying to get at here is that the, the main skill almost involved with getting this kind of data is just standing and operating theater and waiting. And I realized if that's kind of, I mean, you have to obviously think about what task you wanna give them and that kind of stuff, but it's, it's a lot of standing around and waiting.

    And I realize if that's half of my PhD, then I'll come out with skills that are useful for one very particular environment and almost nothing else. Um, 

    Jesse Geerts: so it was kind of a strategic choice there as well. Yeah. 

    Benjamin James Kuper-Smith: Yeah. But, uh, I mean, here's the thing, like if, if. As we say, there were lots of different factors involved, and if like a few of them had been [00:37:00] better or different, then maybe I wouldn't have quit it.

    But the main reason was still that I just didn't like the, the topic. Um, yeah. Interesting. What I wanna say. Oh yeah. So the topic, so the, the point I wanted to make is that, you know, I quit because of a topic. So then when I applied, but I knew I wanted to do research and do a PhD. So when I then applied for other PhD, um, positions, I really, I really paid attention to the project I was doing.

    Um, and, you know, really thought carefully about that. And so in the end, I, I, I, I ended up getting, uh, rejected from quite a lot of positions. Um, and then I saw the position that I'm doing right now and I really wasn't sure whether I wanted to do the project. I thought like, okay, the supervisor seems cool and the methods seem like I'd really learn something there, but I'm not sure that I wanna do.

    Like social interactions and game theory. It just doesn't, I dunno whether I want to do that. Um, [00:38:00]anyway, I, I, you know, now I'm two years into the PhD and I really enjoy the topic, and I don't, I don't, I'm not sure I could have chosen a much better topic. Um, so somehow it's this really weird thing where I, I thought about it for ages, thought like, oh, I'm almost choosing this despite the topic, and now it, it ended up being a great topic for me.

    So the, yeah, the gist of this very long story now is that I have no idea how to like, decide what your topic is, like what it's supposed to be. Yeah. 

    Jesse Geerts: I think that's another thing that, um, it's often sort of overlooked, like when you're deciding for a PhD or probably any job, even outside academia, you're, you're doing decision making with very, very, very limited information.

    Um, and that's just always gonna be the case. So yeah, there, there's a big chance that you'll start something that, that. Will, will be great, but there's also a chance that it won't be quite what you expect it to be. It's interesting what you say about the, um, [00:39:00] uh, uh, about the patients. Um, uh, some people in my lab are currently, uh, having some, uh, getting some data, uh, like single unit data from humans with, uh, epilepsy who have had surgery there.

    But I, uh, I, I hadn't heard yet about this, uh, these very long waits, 

    Benjamin James Kuper-Smith: but that's a bit, that's a slightly different thing though. So if you have these epilepsy patients, I mean, I dunno exactly what your, what your friends are doing, but that's, I think usually something that you implant on a, not a permanent basis, but on the basis of a few weeks.

    Right. Um, yeah, 

    Jesse Geerts: that's, that's true. 

    Benjamin James Kuper-Smith: Yeah. If you have like an ECOG set or in, in your case, single, whereas for direct order simulation you have to be there during the operation. So that's, that's basically I see the big difference between those two. 

    Jesse Geerts: Yeah. That's very different 

    Benjamin James Kuper-Smith: in terms of like practicality of doing research.

    But I mean, so actually, uh, I did learn like one thing though during my, during the, especially the second year of my PhD when I've been doing this, uh, COVID project, uh, 'cause that's a project that [00:40:00] has a lot of variables and it's longitudinal. So it's in a way, a fairly large dataset. That's, I mean, well, I mean, I'm sure there are research project with that are, have way more complex data sets, but I've realized like during the analysis of this project that I really don't like this kind of project where, um, you have a lot of freedom in terms of the analysis.

    I mean, you know, obviously we are careful that we don't, you know, this is like an exploratory study, uh, from the get go. So we're not like pretending that we're looking for spurious correlations there and then make some sort of grand claim out of it. But I realized for example, there, and this maybe relates fairly closely to what you said about the electrophysiology.

    Like, I'm just not gonna do that anymore in the future. Like longitudinal data with many variables. I, I hate to do that. I like, I like doing simple experiments where you have to really Yeah. Think about exactly what I think I like really like theory driven, creating theories, um, and then testing them with fairly simple experiments.

    Jesse Geerts: Is your fear [00:41:00] with the COVID dataset that in the exploratory analysis you might make some kind of conclusions that, that, that, that you could have, uh, avoided if you had some kind of controlled experiment? 

    Benjamin James Kuper-Smith: No, it's not even that. It's, I mean, so, so we, uh. Basically we collected this data like in the first week, I think, of when it was called a pandemic.

    Um, so we were very aware that we just don't know exactly what's gonna be relevant here. Um, and we very like intentionally and openly said like, okay, this is like one question we're interested in and where we have a hypothesis and we're gonna collect all this other stuff around it because it might be relevant.

    We just don't know. So, I mean, like from the beginning, this was a very exploratory study, but for me, what I just don't like is, you know, uh, my, my supervisor, um, often uses the analogy of like different kinds of research you can do. And one of them, and the COVID study is one of this type is you kind of go into the woods.

    Like you, you [00:42:00] have maybe some unexplored territory. You just go into it and you can see, you just see what you find in there. So maybe you go into, I dunno, like the Amazon rainforest and you just see whether you find new species of animals or something like that. Um, and that's one kind of. Research you can do.

    And I just, I've just noticed that what I really like is coming up with a theory and then saying this thing, you know, I dunno, there should this be this kind of species in the Amazon, so then I go and look for it specifically. I've just realized that I really enjoy that kind of process much more in this kind of open explor sheet.

    Let's see what we find. 

    Jesse Geerts: Yeah, there's definitely a, um, a kind of elegance about very beautifully designed studies that test a particular, um, model versus another one. Um, um, and they find sort of like some kind of causal manipulation within the experiment that, that, that, that really distinguishes between the two.

    I, I know what you mean. 

    Benjamin James Kuper-Smith: Yeah. So I think that's maybe something that's actually very useful to think about in [00:43:00] advance because I mean, this is speaking of like. Something that seemed tragic at the time, but is really lucky in hindsight is that I almost, I, I got really close to getting one position where I would've been analyzing large FMRI data sets.

    And right now I'm really glad I didn't get that position. I mean, at the time I was really, it was a bit sad about not getting it because it seemed like it would've been cool. And, uh, I got like really close, but in hindsight I realized I probably would just like completely fallen apart during that project because like, yeah, that's, that's not the kind of work I'm good at doing.

    So I dodged a bit of a bullet there. Yeah. Anyway, um, so back to, uh, back to your PhD program, uh, I have one question. So, which is, uh, so why did you actually apply for that? I mean, did you, did you think you wanted to do electrophysiology and you just realized you don't want to do it? Or did you actually already go in saying something like, I want to do more the theoretical [00:44:00] analysis or modeling.

    Um, and then be close to people who collect this kind of data or, 

    Jesse Geerts: so I was actually hope, I was hoping to do both and have, um, while in the program discussed a few possibilities for doing some kind of, uh, sort of crossover where I'd be doing like the theory work, supporting my own experiments. Um, and I, and there are some people that do that in the building, but it's, it's pretty hard.

    Uh, 'cause you, so if, if, if you are doing theory d driven experiments of like you're mentioning for example, uh, usually it starts with the theory and the experiment follows. So, um, uh, and coming up with good theories, coming up with like, uh, good theories that make testable predictions, like takes a while.

    That can take the, the whole duration of a PhD, for example. Yeah, so, so, so that, that's sort of like one thing. Secondly, as I said, I, I didn't particularly enjoy the [00:45:00] research, so, uh, I quickly sort of changed my tactic and decided to do a main, uh, mainly theory driven PhD. But at the moment that I joined my current lab, uh, Neil Burgess's lab at UCL, I still had the possibility open of at some point maybe doing some experiments, but rather in humans, either something with like EEG or FRI or something like that, or possibly behavioral as well.

    Benjamin James Kuper-Smith: I mean, in, by the way, this is a question I've always had, is it Burgess? Or Burgess? Is Legacy Burgess the emphasis on the first or the second half of the, of the name 

    Jesse Geerts: on the first 

    Benjamin James Kuper-Smith: Burgess. Okay. I, I think I always got that wrong then. Um, but yeah, because uh, this is actually another, so how did you end up.

    Doing your PhD with him, because I mean, so there's a FA few interesting kind of points here. The first is, of course, that we both took his module when we did our masters. The second thing that slightly confuses this though is that, I mean, [00:46:00] he, as you said, does mainly stuff in humans, if I understand that correctly.

    And is even part of the, uh, Sainsbury welcome or is he, he an external, like how exactly does that work? 

    Jesse Geerts: Yeah, so, um, Neil actually has a rodent part of the lab as well. Um, so his, his lab is really big, um, and spans currently three different buildings. There's the Institute of Cognitive Neuroscience at Queen Square.

    Where most of the, uh, human, uh, cognitive neuroscience happens. And then there's a part of the lab, which is in the anatomy building, uh, with rodent work. And, um, there are a few postdocs, uh, in the, uh, Sainsbury Welcome center as well, working on rodent stuff. 

    Benjamin James Kuper-Smith: But you didn't do a rotation with him, or 

    Jesse Geerts: no? No, I did not.

    But one thing that appealed to me about his lab was this very fact that his [00:47:00] work is pretty translational between these different systems. Um, there's, there are people who are just doing like pure theory work. Um, there are some people that do, for example, uh, I just mentioned this, uh, study in human epilepsy patients that is led by Dan Bush, a senior postdoc in the lab who is mainly a theorist but does like yeah, experimental work with humans as well.

    Um, then there's also, uh, the rodent stuff and looking at sort of these same questions that our lab is interested in, which is in the domain of like memory, um, representations of space, et cetera. Um, looking at those same things and trying to find those same, uh, representations in humans and rodents is pretty interesting.

    I think for example, in, um, the sort medial temporal lobe in, uh, rodents, you find these play cells and, and grid cells, et cetera. And, um, [00:48:00] there's been some FMRI working in humans where, uh, in our lab, which, where you can see evidence of these grid cells, uh, in FMRI as well. 

    Benjamin James Kuper-Smith: Yeah. Is it then like on a like kind of day-to-day or maybe weekly basis, is that like a lot of exchange between the people in this lab or is that more like a theoretical thing where you actually mainly talk to.

    The people who do the theory and then like once a month, once a month or something, you see someone who does electrophysiology or how does that work in practice? 

    Jesse Geerts: Yeah. So that is like one thing that I, I, I expect it to happen more when I joined the lab, I must admit. Um, and so, so yeah, when I, when I first joined the lab, we didn't actually, uh, I was surprised to find out that we didn't ever actually have a set lab meeting.

    Um, so the lab goes out for lunch, um, very often together. Uh, so that is, that is sort of like an informal lab meeting, but there's no, [00:49:00] uh, set lab meeting where there's an agenda and, and we go through all this kind of stuff. So any kind of, uh, crosstalk between the different researchers in the lab, uh, happens kind of organically rather than it being, um, it being, uh, set up.

    Uh, specifically, uh, unless Neil notices, uh, oh, you're working on this thing and it's actually related to what, uh, this other person in the lab works as well. So I'm gonna, uh, put your guys together, but there's no, there's no, um, there was no lab meeting, which, which, which I think 

    Benjamin James Kuper-Smith: is that an intentional part of his, or like, because like, you know, I've, I've fairly, I'm not sure I know of a lab that doesn't have like a, a centralized lab meeting.

    Um, was that like he specifically didn't want that or? 

    Jesse Geerts: Well, yeah, I'm not really sure. Um, so we, we currently do have one, uh, twice a week during the pandemic. Um, and I think it has to do with Neil's, um, Neil's supervision style a little bit, where like [00:50:00] he's quite, um, let's say fair. So he will, he will just.

    He, he, he gives you a lot of freedom to, uh, go whatever path you want to, but you check in with him quite often. Um, so, and he's always, he is, he is often very at the lab himself. He is very present, so he is always there for you to ask him, uh, questions. But it, there's no, there there's very little him pushing you.

    Like, like, like I think, uh, other supervisors, 

    Benjamin James Kuper-Smith: I'd imagine that's kind of standard in a big lab, right. From what I kinda understand is that if you have like a, like as, as, as soon as the lab is more than like 10 people or something, then you often get, um, the postdoc supervising the PhD students more on a daily basis.

    Um, 

    Jesse Geerts: yeah. Well and that's, and 

    Benjamin James Kuper-Smith: that's definitely how goes my impression of kind of lab sizes. 

    Jesse Geerts: Yeah. 

    Benjamin James Kuper-Smith: Um, so who were you working with then? Mainly like in the last few years? 

    Jesse Geerts: Um, so I actually don't, didn't have one of those, uh, postdoc [00:51:00] supervisors myself. But I have, there are some, uh, some more senior theory postdocs, uh, in the lab that I sort of often bounced my ideas back and forth with, uh, such as Dan Bush himself.

    Um, Andre Bisky, um, is another one. And, um, I recently graduated more senior PhD student at Tal and Evans as well. 

    Benjamin James Kuper-Smith: So is, I'm wondering, just because the, both of the, uh, the, the, the one Preprint I have here and your paper are both also with Kim, is she, but she's not even, I mean, she's at Deep Mind, right? So how does, how does that exactly work?

    Jesse Geerts: Yeah, so, um, 

    Benjamin James Kuper-Smith: because also I still don't understand how deep, how DeepMind really works as a company. Like, like what the people, people actually do that work. They're like, it seems they're all doing a lot of research, but it's a company, so I don't know. 

    Jesse Geerts: Yeah. Um, yeah, no. Um, so Kim, uh, has been sort of, um, informally [00:52:00] supervising me from quite early on in the PhD.

    I know her through a, uh, common friend, uh, who is in, who, who used to be in my, in bridges, new Berg's lab as well. Um, I told, uh, this is Sophie. Uh, I told Sophie I was, um, uh, interested in these or reinforcement learning theories of hippocampus. Um, and she said, oh, you should talk to Kim sometime. So that's when I started like regularly chatting to Kim about, um, some ideas and, uh, and my project.

    And at some point she said, oh, would you be in the market for a bit more supervision on the RL side of things, the reinforcement learning side of things of your project at that moment? Uh, yeah, I was quite at the, the beginning of my PhD, so I was like, I was quite grateful for that. Yeah. Ever since we've, we've had a, had this really nice collaboration going on where, um, I, yeah, I, [00:53:00] she, she, I, I bounced back and forth my ideas, um, like every, every two weeks or so.

    And, um, uh, yeah, and we, and we've written, um, uh, both of these papers together actually.

    Benjamin James Kuper-Smith: It sounds, yeah, it sounds pretty cool. I mean, I mean, also, yeah, just because of the stuff she did previously, especially the Nature neuroscience paper. It must be pretty cool to kind of have her look over your shoulder, make sure you, you, you do the right thing. 

    Jesse Geerts: Yeah, she's, yeah. So, so she's great. Kim is great.

    Uh, she is, uh, really very smart. Uh, and on top of that, also really very friendly and approachable. Um, so this is, it's, it's been really nice for me to have someone like that. And I think. If, if I hadn't started talking to Kim at the start of my PhD, I would've probably sought out, um, a little bit more of formal supervision from one of those postdocs in the lab.

    But [00:54:00] yeah, from really, from really early on, uh, in my PhD, Kim really filled in that role. So between, uh, Kim and Neil, I've had, uh, um, really all the supervision that I needed. 

    Benjamin James Kuper-Smith: Sounds pretty good. Uh, I have one last question about the PhD program, kind of like, not specifically to the paper, which is, um, just a more general question.

    Like is there anything that you, how should we say, wish you would've done differently during your PhD, um, or for anyone who's starting your PhD? Some sort of advice, I mean, some errors I guess you have to make yourself. Um, but I'm sure there are some things, or maybe there are some things, uh, where you go like, yeah, don't, I dunno, do or don't do X or y.

    Jesse Geerts: Yeah. Um, so I, I think my, one of my common pitfalls, uh, and I, I would love to say that I've already, uh, learned from that and I never do it again, but I, I still do it, is that, [00:55:00] um, if I do not understand something, I have a sort of, sort of, sort of reluctance to seek out help. And, uh, I think I, you know, I, I sh I should be able to understand this, so I'll, or I I'll, or, or I should be able to work this particular problem that I'm supposed to work out, out mys by myself.

    Uh, and then I, um, go on that solo for what, what feels like, uh, like a while, but then at some point you, you, you look on your calendar and you've been working on it for like six weeks straight and you've just wasted them on nothing. Um, and that's, that's, I think a, can be a very. Good thing sometimes, but you should not do that too much because, um, it's really fine to ask for help and it can really speed up your project sometimes.

    Benjamin James Kuper-Smith: Yeah. I find that a really difficult trade off to make that, because, you know, the whole, the whole thing about doing research is often figuring [00:56:00] stuff out on your own and doing something that, or if you wanna do something that no one has done before, you kind of need to be the person who figures out how to solve the problem, how to solve the problem.

    Um, so. It's kind of, you know, in a way that's necessary. But I also, and that sometimes also, like I, I've, you know, kind of in some sense reinvented the wheel once or twice in my PhD and in a way you could, you know, say like, well, that's kind of a waste of time. But I, I did feel like it was really useful to have done that.

    So, but then as you say, like you can also just waste huge amounts of time by just running against the wall repeatedly. 

    Jesse Geerts: I think it really depends on your personality, I guess. Like, so the, the, the advice I just gave was probably more an advice, uh, for myself than for anybody else. But like, so if you're, if you're naturally someone who, who is prone to going for something or like, is, is not so prone to asking questions, uh, like myself, then, um.

    Then this is a pitfall to look out for. On the other hand, uh, if you're [00:57:00] very prone to always asking for help and always wanting someone to hold your hand, uh, then you should probably go the other way, uh, a little bit because your supervisor's gonna find you very annoying.

    Benjamin James Kuper-Smith: Yeah, that's a, that's a good clarification there. Um, yeah, I mean, it's, I mean, this is the 

    Jesse Geerts: annoying thing, right? I, I have a friend actually who has a student. She's, uh, a friend of mine supervises the student remotely for a while, and, uh, the guy was a, um, uh, a machine learning student in Stanford, but, uh, he, he, he emailed her, uh, with questions like.

    How do you plot not one, but two different lines in that lab, you know, in the same, on the same axis. So it's sort of the ultimate laziness. So don't be that either. Be somewhere in the middle. There's, 

    Benjamin James Kuper-Smith: I mean, that's, that's the criticism [00:58:00] of, of machine learning right, though. Just be told what the answer is and then repeated.

    So I guess in a way he is taking his machine learning very seriously, but Yeah, yeah, yeah. Don't, don't ask, just Google it. Someone would've asked that specific question on Stack Overflow like 10 years ago. 

    Jesse Geerts: Yeah. Yeah. So don't ask Googleable questions, but, but, um, yeah.

    Benjamin James Kuper-Smith: Okay, cool. Um, so, um, I'd like to move slowly towards actually talking about the paper. Um, maybe how did, okay, so you did your rotations realized you don't wanna do electrophysiology. Somehow you ended up in Neil Address's lab and, uh, supervised by Kim, uh, on a, uh, fortnightly basis. Um, so how did, so [00:59:00] talking about the p and a s paper, how did, uh, you kind of like, what are the beginnings of that paper?

    I'm assuming it wasn't just, I want to create this model of hippocampal and dorsal strata learning and decision making, and then you just wrote it. Um. So how, how that start? 

    Jesse Geerts: So this was actually me joining a, um, existing project, uh, that was already going on in the lab. Um, it was started by, uh, Fabian Ey, who is my, uh, author on the paper, um, who, uh, was a postdoc who le left for industry, um, a while before I, uh, joined my PhD.

    And, uh, he, he left an unfinished project basically that, um, Neil said, uh, why don't you start working on that? It could be a great introduction to, um, to learning some theory. When I, um, when I started on project, I, uh, was told there was a code base with some code for [01:00:00] some of these, uh, simulations, but, um, none of the.

    Uh, I, I couldn't actually find one of, uh, uh, any of the code, and Fabian himself was very busy at that moment, so it was pretty difficult to get ahold of him. Um, so yeah, as a first exercise, I, I, I, I kind of like started over at that moment. Um, there was some ideas about these like, um, al model free controller that is still part of the, uh, paper.

    Um, but the, the hippocampal controller, which is also part of the paper, um, is uh, was conceived as, um, a allocentric learner. Uh, but not necessarily this, uh, successor representation model, uh, that we currently have. And a along the way I started finding that like a, a sort of model based type approach for hippocampus would fit the data better, um, than the original [01:01:00] model.

    So yeah, the way I started with this project was jumping on a project that was, uh, already going on and giving my own, uh, giving my own twist to that. The, the idea for this successful representation model, uh, which I can explain, uh, what it is in a bit if you want, of course, came, uh, from talking to Kim about this, who, um, has a previous paper where that is introduced.

    Uh, I think you mentioned it already, this nature neuroscience paper where she introduces that as a model of hippocampal play, self firing. 

    Benjamin James Kuper-Smith: So what was the kind of goal, or what were you trying to do then when you, like, once you joined the project, was it to model, to find one model that can combine spatial navigation literature and decision making literature or what exactly what you're trying to do when you were doing the simulations?

    Well, 

    Jesse Geerts: yeah, so there's a, there's a bit of, um, there, [01:02:00] there's these two different fields, uh, the, the reinforcement learning field and the spatial navigation field that have some overlap. And they both talk about these different, uh, learning strategies in the reinforcement learning literature. People talk about this model free versus model based reinforcement learning where, um, model free reinforcement learning or, or, or habits.

    Or just learning from your, uh, from the rewards and the mistakes that you got. Just like, I'm gonna repeat actions, uh, that were, that led to reward in the past, basically. And a model based planning system. Build some kind of model of how does different states, uh, in the world relate to each other and uses that model to simulate the future, uh, and, and, and, and tries to get to an optimal policy in that, in that kind of way.

    Now, that seems to al almost directly map onto the different strategies that you see in spatial [01:03:00]navigation. Or, or at least like, they're often, they're often talked about in a similar way, uh, where you can have navigation. So navigation, they, they make this, uh, distinction between response learning on the one hand and place learning on the other hand, where response learning is defined as remen, remembering an egocentric or self-centered, uh, sequence of turns, for example, like, uh, go, uh, left at the tree and then right at the church or whatever.

    Whereas place learning would be viewpoint in variant, like remembering a location, like remembering a location on a map or whatever. So regardless of where you're coming from, uh, for example, the, uh, the treasure is buried at, uh, five meters east of the church. And, um. Now that, so that seems to map on really nicely because this, this cognitive or this map based, or like where, where if you have an inter internal representation of a map, uh, that could be your nice ground [01:04:00] for the, um, for, for, for this model, for model based reinforcement learning.

    But, but it's not necessarily, it doesn't necessarily map on one-to-one because in principle you could do planning in an egocentric space, for example. So you, you plan to first go left and then to right and so forth. And it can be, you know, done in a very flexible manner. Uh, and you could also, um, have a sort of map-based representation of the environment and do a model free reinforcement learning over that map where you just have a model free association between.

    My, this place and a and a reward. So a, so you just have a, you think I'll just go back to this place to get that reward every time, which is, is this a sort of like inflexible model free strategy, but apply to a center representation? Um, so, so, so the goal of the project is really to set out these different things and see if they map onto each other.

    So yeah, in the paper we, we simulate a bunch of [01:05:00] behavioral experiments, uh, using this model based and model three different controllers where, um, and we show indeed that like, uh, having a, uh, some of these placed learning behaviors, uh, yeah, that's sort of like, yeah, I, I would say if I would, if I would like summarize the findings of the paper in one sentence, that some of these placed learning behaviors, those Allison group behaviors are indeed.

    Captured only by a model, model based like strategy, the success representation or model based. We don't really distinguish between those two. But like you need to have a non, a, non-model free, uh, like strategy. The, the animals follow a non, a model free like strategy when they're doing these place learning behaviors.

    So, so, so indeed, indeed, uh, the egocentric behavior seem to be, uh, quite model free in these tasks that we simulated and, uh, the, um, centric like behaviors, [01:06:00] uh, seem to be model based, which, which, which, which conforms to, uh, an intuition that many people had. But, uh, our paper is really sort of fleshing that out a bit more, a bit more precisely 

    Benjamin James Kuper-Smith: because, so this is like one kind of big question I have about this.

    So. So, you know, like I, I read, I read the paper and thought like, okay, this sounds really cool. Like you have these two separate things that are um, can be kind of explained by the same thing. Um, but then I did immediately have this kind of, not exactly criticism, but this thought of, well, yeah, but isn't this kind of what everyone has been saying already?

    Or, um, like the, the, in the immediate example I thought of, and I think you might even be using this in the introduction, I can't remember, um, but is that reinforcement learning is often explained with an example of spatial navigation. So you [01:07:00] use something like, can say, model-based is, you know, you have a map, like a map, right?

    That's, that's the whole thing with the play cells, right? You have a map and then model free learning is, is explained with a habit where you just go left or right. If that, uh, letter brought in the past. In the past. Um, so like one thing that I. I mean, in a way it sounds like a very critical question, but it's just, I just don't quite understand this.

    Uh, is that like the, the question I had was, so what exactly are you kind of then adding if it, it seems to me, you know, like that this, this analogy between these two things was how I almost learned about them in the first place. So, do you see what I mean? Like, to me it's not entirely clear exactly what the model adds beyond what, um, what's already known when we learned it.

    Jesse Geerts: Yeah. So, uh, I mean in like, in principle a great point, um, I think a lot of these sort [01:08:00]of intuitions were already there. Still, though an, an intuition is not quite the same as, as a computational model of course. Um, and, and fleshing out how you would actually do this, uh, arbitration mechanism on, based on reliability is one of the, of the.

    Uh, contributions here. Um, but the second contribution I think is the fact that you have, like, there seems to be a nice intuitive, uh, understanding of, like a map is like sort of a cognitive map is a bit like a model. As in model-based reinforcement learning is, is not quite right because of the point I made earlier.

    Because the, the map can be seen as sort of the representations that you learn over sort of like where, where do you use a map or just like learn directly from cues, uh, is sort of representation you start with and, and how, how you then like [01:09:00] associate that map to reward is, is another question. So, uh, we're really sort of like, uh, saying like there's a.

    Difference between your navigation strategy that you choose and the decision making strategy that you choose. Um, and in the brain they happen to, uh, uh, map onto each other, but computationally that is not necessarily true. 

    Benjamin James Kuper-Smith: Sorry, can you, maybe I misunderstood something there. 

    Jesse Geerts: So, so it, it, it makes a lot of sense that the brain, uh, uh, works in that way.

    It's because to build this sort of map like representation, you need to have some kind of model of the world. You need to know how different locations relate to each other. But it's perfectly possible that you first build such a model and then, and then uh, you perform, uh, some model free incremental type reinforcement learning over that.

    And, and, but that's not what we see happening in the brain, and we see that indeed, indeed, if, if there is such an centric representation that an animals use indeed at that moment, they're also more [01:10:00] flexible with, uh, in their behavior. Uh, for example, uh, when the. Rewards that they work for are devalued.

    They change their behavior when they're being centric, but not when they're being egocentric. 

    Benjamin James Kuper-Smith: Okay. Can you, um, it seems like maybe I misunderstood something about the paper then. Can, can you maybe explain, I don't know whether we can just use a microphone example here or maybe use one of the simulations here, for example.

    I think the, the, the water maze is something I also feel like I did not quite understand something about, but could you maybe explain like with some slightly more concrete example and maybe a bit of an anthrop, more izing of a rat, like how these mechanisms work? 

    Jesse Geerts: Yeah. So basically if you, for example, think about, um, so one of the famous experiments that people use to test this, like map-based learning versus response learning is a, uh, plus shaped maze [01:11:00] where.

    There's four arms basically. And, um, animals are trained to, for example, always start in a north arm to then run to the west arm to take a right turn to the west arm. And then on probe trials, they put the rat in the opposite arm and they, and they check whether do you make the same ecocentric turn or do you take the same, uh, going to go to the same allocentric place, so taking a left turn instead of a right turn.

    Yeah. And what you see is that animals, uh, early on choose this, uh, are in this sort of map based strategy where they go to the same centric place if they're, if you give this probe trial early on. Um, but if they are overtrained, so they've really done like lots of trials, they switch to an egocentric strategy now that, that could be, so this is, this is a good moment to, um, make the distinction that could be explained by any model.

    That, [01:12:00] and this is often sort of like referred to in a reinforcement learning way, but it, but it could be explained by any model that uses any sort of, uh, so it's basically the pla the, the place learning strategy doesn't need to be model based. It can, it can be just, uh, any model that has an allocentric representation of that environment to be, uh, uh, they can do model free learning or model based learning over that would choose that play strategy.

    And any model that has an egocentric, uh, representation, be it model free or model based, would choose the egocentric response strategy. So, so, so the, the, the, the, the fact that you see, for example, that if animals have a hippocampal lesion, they only do the, uh, response strategy. And if they have AAL dorsal lesion lesion, they only do the play strategy that doesn't, uh, yet tell you the learning strategy that they have used to learn.

    Learn about reward [01:13:00] there. So in uh, in the paper we put right next to that a devaluation study where if you devalue the reward, uh, you see what happens. And now that, that, this is sort of a classic thing in model based versus model free learning, that a habitual controller, and that this is, um, this is conceived as model three.

    Reinforcement learning will keep on working for a reward, even if the, even if the animal doesn't like that reward anymore. So in these experiments, the reward, for example, is paired with an illness or maybe, uh, they've, they, they, they just give so much of the reward that they're not hungry anymore or whatever.

    And animals that are really overtrained and the theory says, oh, these are model free. They keep on working for that reward even if I, if they don't, uh, want the reward anymore. And now what you see is that, that in this plus maze, this. Placed learning strategy. So the Allocentric one is indeed sensitive to reward learning, [01:14:00] but the, the response learning one is not showing that it's indeed a more, that, that indeed the allocentric strategy com, um, corresponds to, uh, sort of model in the model based model, free reinforcement in any kind of sense.

    Um, and so this is, this is sort of one example that we, that we highlight. Um, but we, we, we basic basically show a series of experiment that point to such a distinction.

    Benjamin James Kuper-Smith: Okay. This is, I think, uh. The first time on the podcast that I realized I really misunderstood the paper, uh, somehow I completely missed some of the points you made. Oh, right. Okay. So this is, I'm not even sure what to do right now because I feel like I just have to read the entire thing again. I think I somehow 

    Jesse Geerts: that's, that's probably my fault for writing the paper not clear enough.

    What, what, what, uh, what is, um, the mismatch between what I said and what you thought?

    Benjamin James Kuper-Smith: Um, I mean, so just as a caveat, as I said, [01:15:00] like my productivity has plummeted this month, this particular, so 

    Jesse Geerts: yeah, 

    Benjamin James Kuper-Smith: I, I read only roughly the first half of the paper. Um, yeah. Yeah. So, um, so maybe if I'd read like a hundred percent of the paper, then it would've been a bit different. But somehow I thought the, um, I think I, I somehow thought the claim was that basically.

    Yeah, that, that these two things are just the same thing almost. That you, if you have, um, model-based reinforcement learning that that is done via placed us in the hippocampus. Um, you know, almost saying like that, this analogy is just a perfect one-to-one mapping almost. 

    Jesse Geerts: So, so yeah, we're saying, I don't, I don't want to quite say once more mapping, but we're saying in the brain.

    Yes, indeed. Um, uh, the point I was making earlier is that mm-hmm. Like the, like indeed, like we're saying in the brain, these [01:16:00] things do seem, uh, to map on really well. Um, and there has been some contr controversy, um, uh, there because the, the, the, the hippocampus like historically has been sometimes not found to be, uh, involved in this kind of devaluation paradigms.

    Um, and sometimes it has been. In spatial type tasks. Um, the hippocampus is always, it, it has always been, uh, in, in found to be involved in this kind of like allocentric type navigation, but it in a sort of like lever pressing task, uh, where the animal just in some kind of reward box and needs to press a lever for a reward if they, if you do this kind of like, uh, devaluation tasks, the literature is a little bit hazy on that.

    More recently though, um, there have been some, there's been some experimental work both in humans and in [01:17:00] animals where the, where, where they've basically shown that in this, there there's this specific two step task, uh, designed by Nathaniel do and colleagues, uh, where, uh, which is basically a continuous reevaluation paradigm where on every trial the rewards are devalued and.

    Uh, you have to or are revalued and, uh, the, on a sort of trial by trial basis, you can estimate how model based or how model free you are. And on that task, uh, it does indeed seem that, um, the hippocampus is involved in both, um, road and St. Humans. But yeah, so yeah, the answer, the answer is a little bit complicated.

    Benjamin James Kuper-Smith: Okay. Yeah. May, maybe I made it more, uh, yeah. Seem more clear cut and black and white than you actually wrote in the paper. Oh, to be fair though, I have to say, like, as I said, like when I read the part about the Morris Water maze, there was a, I did somehow, like when I read it, you know, you read something, okay, yeah, I see where this is [01:18:00] going.

    And then somehow went the opposite way of what I thought you were gonna say. So maybe I'm just, uh, maybe I also just think I understand these paradigms better than I actually do. 

    Jesse Geerts: Yeah, so the Morris Water Mae task, um, is a purely spatial one here. Um, and in this version of the task, so that there's a sort of standard model, Morris Water Mae task, which is just about, uh, having a, um, animal swim around a, um, circular arena where there's an opaque, um, water in it and there's some kind of some hidden platform somewhere, and they need to find that.

    Um, in this particular task, the hidden platform was indicated by a local queue that was, uh, placed 20 centimeters, uh, next to it. So, and then the landmark and platform were moved together, but their relationship was, uh, kept constant, uh, in every different session. Now, the reason for that was that they wanted to tease [01:19:00] apart this kind of like Q based strategy, which they think is sort of.

    More like egocentric, sort of like go, go right at the queue from a sort of place like strategy where you, um, where you, where you, um, remember the exact location of that platform, um, relative to maybe like digital queue somewhere. So they're in a more sort of, in a, in a, in a more like spatial memory type way.

    Now. 

    Benjamin James Kuper-Smith: Uh, sorry, just so I, uh, know exactly what we're talking about, um, with this kind of task. So the, there's no queue, like, uh, there's no queue. So I'm wondering like, what does the animal see? Like is there any kind of queue outside of the arena that they can see, or is it literally just they see kind of a, let's say like blank arena and in the middle there's somewhere there's gonna be a queue.

    And then, um, so other rats 

    Jesse Geerts: there, there's [01:20:00] there they have some very distal queued uh, cues which are just, uh, really far away, which is what the place cell map is anchored to. So you can show this by, for example, by example, for example, rotating all those, uh, spatial cues and the, and the, and the sort of place cell map as you recorded, uh, rotates with it.

    Benjamin James Kuper-Smith: And do they, you always put the rat off from the same point, or is it put in random points in the auto automat? 

    Jesse Geerts: Um, I think it's point put at random points. I'm not a hundred percent sure, but yeah, random, I think. 

    Benjamin James Kuper-Smith: Okay. 

    Jesse Geerts: But, but the, but of course the, uh, the edge of the arena itself is kind of a lo kind of a local landmark as well.

    So there is a, there's a, there's a sort of like edge. Yeah. But that looks the arena, like the wall that looks 

    Benjamin James Kuper-Smith: the same everywhere 

    Jesse Geerts: it looks, it looks the same everywhere. Yeah. But if you have the edge of the arena, and, 

    Benjamin James Kuper-Smith: uh, I guess what I'm, so I'm trying [01:21:00] to imagine right now, just from a, like, you know, in quotation mark, participant's perspective, um, what it would actually mean to do the task.

    So I guess I know that there's a platform somewhere that's linked to, um, or there's a, there's this landmark and close to that, there will be this platform and I can go on that and then I don't have to swim the entire time. But if you change both where the animal is located and where the landmark is not even sure how.

    Jesse Geerts: So a strategy of just swimming towards the landmark and swimming a circle around the landmark and at the distance of the landmark, where you've always found the platform will, will get you to the platform. 

    Benjamin James Kuper-Smith: But how don't the animals know every time that they just have to swim, 

    Jesse Geerts: uh, to the landmark. So they learn that by experience.

    So [01:22:00] there's a few, um, things to like take away, like of course in the, in the first trial ever that the animal does that, it has no idea. It's just panicking. And it doesn't particularly like swimming. So it's gonna do some random exploration the first time. It then hits the platform. It can do two things. Uh, it could, in theory do two things.

    It could think like, oh, well at this place in my cognitive map, there is a, which I've anchored to these very distal cues far away. There's a, there's a platform. So next time I'll, I'll just go back to there. Uh, from wherever I am. Another thing it could do is like, oh, well, I'm currently 20 centimeters south of the, of the local landmark, so I could use that as a queue.

    Now, um, in this particular experiment for four trials long, that was, that landmark was kept there, so it could just sw swim back there, but then after four trials in the next session, the landmark would be at [01:23:00] a different spot. And then the animal has a choice. Do I go back to the same place I was before? I can still distinguish that from these distal cues, or do I go to the local landmark?

    Now that what you see is that most healthy animals, actually in that second session, most health healthy animals actually swim back to the same place. Not, and they don't follow the local cue, however. If you keep on doing this every, uh, every time four trials, it's at the same spot and then you move it four trials at the same spot and then you move it and so forth.

    Then more and more that local queue becomes a more reliable indicator of where the, uh, landmark is gonna be. Um, or sorry, where the platform is gonna be. Um, and you see that animals start following that more. 

    Benjamin James Kuper-Smith: Yeah. But only if they're healthy. The lesions ones, I mean the lesions ones, it doesn't even really, or it doesn't really matter which trial it is.

    [01:24:00] Right. Whether it's trial one or trial four, that's pretty, 

    Jesse Geerts: so would you see, uh, in, in, in, in lesion animals? Is that the sort of in intercession learning that sort of like learning that particular place is almost completely abolished, but across sessions, learning the relationship between the landmark and the platform that's kept intact.

    And so over sort of the total of 11 sessions that they do, they actually perform as well as LP animals, which. Points to that there is some kind of extra hippocampal controller or sort of extra hippocampal system that that, that does this landmark based navigation. So yeah, this, that, that study by, um, Pierce and colleagues was really aimed at sort of trying to try, try, trying to tease apart these different paradigms of, of la of of, of, uh, map based learning and uh, Q based learning.

    Benjamin James Kuper-Smith: Okay. So can you then again, just briefly explain how then [01:25:00] the two types of reinforcement learning and the place learning versus response, learning how those then map onto the behavior? 

    Jesse Geerts: Yeah, exactly. So, um, now in this particular case, the, there is no, in this study, there is no direct testing of. No very specific testing of model based versus model three.

    Uh, not, not explicitly at least. So there is really just a test of place learning versus response learning. In our model, the model free system learns through direct associations between a set of, of represe egocentric representations of these local landmarks and value. So it's just gonna like action values, which correspond to movement directions.

    So they're really just going to move in a direction that was rewarding in the past based on these, uh, based on what the landmark is saying. [01:26:00] 

    Benjamin James Kuper-Smith: But, sorry, just, um, uh, one brief question here. 

    Jesse Geerts: Yeah. 

    Benjamin James Kuper-Smith: Um, you, you. To me, it, it would seem as if, so you have these two ways that you can anchor to which you can anchor your, the reward, right?

    One is the, the distal cues, whatever big thing that is in the room, and the other is the specific landmark in the environment. I'm kind of surprised that, I think you said earlier that the animals initially go for the more distal cues. I'm kind of surprised, I would've imagined that they immediately go for like, well, it's next to that thing that's, you know, right there.

    So, 

    Jesse Geerts: yeah. That's, that's interesting. It's interesting. It's something you find in. It's something you find in general in the spatial navigation literature. So in the, in the, in the other study that we just discussed, in the plus maze, you find that too, the, the allocentric strategy comes first and they move to an egocentric strategy only through habit formation, really, sort of like, oh, it's been, it's been this, [01:27:00] it's been the case so often.

    This is probably not gonna change. So I'll, I'll, I'll, I'll just do that. And what you see then as well is that these animals then become inflexible in their behavior as well once they move to the egocentric strategy. How well that, how well that translates to humans? I'm not entirely sure. I saw a study presented once by a navigation researcher and in humans, and she showed something that like, um, people who grew up in, uh, Sheena, the, um, Italian city, uh, versus, versus so, so people who grew up in Senna, where, which is characterized by.

    A quite narrow little alleys. Um, and, uh, quite high buildings, they often will be very prone in a similar sort of study to this one. But in vr, uh, they will be very prone to taking a sort of egocentric root learning strategy. Whereas people who grow up in American [01:28:00] cities, I think they took like, I don't know, I, I dunno, Atlanta or something, whatever, uh, they're prone to using this, uh, kind of allocentric strategy.

    So there's, there's some learning involved there, I think. But yeah, animals often, uh, first when they're put in this kind of, um, experiment, they, uh, they, they prefer a, um, robust strategy. I think in general, uh, far away distal cues on the horizon, sort of projected at infinity tend to be, um, tend to be, um, constant quite a lot.

    Like, for example, mountains and stuff. Whereas, um. It could be that your local queues are subject to more change. I, I'm not entirely sure whether that's true. 

    Benjamin James Kuper-Smith: I see. Okay. So is it almost the case that like if you don't know the environment you are in, um, you don't even know what variables are changing within it?

    The kind of, uh, so, you know, they're not very reliable queues in that sense, but [01:29:00] that, uh, yeah. Large stuff like where the mountains are, that's not gonna change. So at least you have something that's very reliable to use. Is, is that kind of the logic here? 

    Jesse Geerts: That, that's, that's my hypothesis. Um, we haven't done any, I, I wanna make it clear that we haven't done any research, uh, on that.

    But, um, it's something that we observe, um, that we, we, we observe that this is true. Animals prefer a place planning strategy when they don't know the environment yet, not that well yet, and they move to an egocentric strategy la later on. I could imagine that, but this is really just a speculation. I could imagine that, um, in nature, faraway queues are often a bit more reliable because of this sort of like continuity of mountains versus maybe a tree which could be gone the next time you're there because.

    The lumberjack took it away. 

    Benjamin James Kuper-Smith: Lumberjack [01:30:00] just came and chopped down trees in your garden. Although I could imagine for a mouse, it's, the tree is probably the distal queue, right? Like the, the mountains are so far away they probably don't even see it. Um, 

    Jesse Geerts: yeah, sure. Yeah. I, I'm, I'm generalizing to like a, a sort of like general sort of natural foraging setting 

    Benjamin James Kuper-Smith: to humans, 

    Jesse Geerts: I 

    Benjamin James Kuper-Smith: guess.

    Jesse Geerts: Yeah. 

    Benjamin James Kuper-Smith: Yeah. Okay. I think that what I've really learned is that I really need to read this again. I think I need to, yeah, I think, uh, yeah, I also see, I dunno how much of this I'll keep in. I think a lot of the last half an hour was me just not quite understanding the paper. That's alright. How useful that is for anyone to listen to.

    Jesse Geerts: I'm, I'm, 

    Benjamin James Kuper-Smith: I'm happy to explain it. We'll see. Maybe I'll keep it in. Um, 

    Jesse Geerts: to be honest, I, I didn't actually realize that you had, uh, had read the paper. I thought, I thought you were just gonna ask me about it. So, um. That's more, that's more preparation already than I expected. 

    Benjamin James Kuper-Smith: Uh, well, I mean, no, usually I actually prepare very, I dunno how much that comes across, but usually I [01:31:00] prepare quite a lot for these podcasts.

    Um, usually I actually have read like a few papers by the people and understand them. Uh, I think that the main problem here is just that, uh, there's stuff like reinforcement learning is stuff that I've kind of heard about for quite a while now. So I probably think I understand it better than I do, um, because I've, I've never actually really like sat down and worked with it and all that kind of stuff.

    So that might be part of the problem that I, I feel like I probably, yeah, I, I think I, I feel like I understand these things, but I'm not sure. Yeah. I've only, you know, read a few papers that use it, maybe saw a lecture, an introductory lecture on it or something. And that was like two years ago, so. 

    Jesse Geerts: Hmm. 

    Benjamin James Kuper-Smith: Um.

    Yeah. No, no. I, I wanted to, I plan to actually have understood the entire thing, and I kind of, the, the, the goal is almost that I can ask the questions of the things that I can't [01:32:00] get from just reading the paper. You know, like, for example, what your, um. You know, like the, the, the stuff that we talked about also a bit of the, in the beginning, like where it came from, why you did certain things.

    But I guess with this paper, um, with some of those questions, I'm lacking the basis of actually understanding it. 

    Jesse Geerts: Well, you can just cut out all of those, uh, things where you didn't understand it and you cut something in sort of like asking a very critical question. And then you have me mumbling, I dunno.

    Benjamin James Kuper-Smith: Exactly. I'll, I'll edit the entire thing to make you sound like an idiot. Yeah. You actually asking, just, I'll just add, had questions about something and completely else's, like, oh, so how's life in London? And then you just keep, you just keep talking about this topic like a maniac. Um, that might be a good strategy.

    Uh, we'll see. Yeah, that's the, yeah, that's the weird thing. I, in, in a sense, I have a lot of power here because I [01:33:00] can just, 

    Jesse Geerts: you do, 

    Benjamin James Kuper-Smith: and this in a way to. It seemed like whatever. I want to make it seem like there's just a slight caveat that if I do that, I think I'll, I'll very quickly run out of guests who want to be on the podcast.

    Um, so yeah, 

    Jesse Geerts: it's like Ben's roast. 

    Benjamin James Kuper-Smith: Yeah, exactly. Me just shitting on people who don't even know what's happening. Um, I'm just, I'm just trying to get you to talk so I can just get material that I can use, use against. You're like, 

    Jesse Geerts: listen to this idiot. 

    Benjamin James Kuper-Smith: Exactly. Yeah. Actually, yeah, I have two podcasts. One is the one where I release the actual conversation and another where I just chop it up.

    Um, it's like, guys, I find another moron. Uh, yeah. Uh, so one thing I wanted to ask about, which is, which relates to more kind of, um, broader modeling kind of questions is, um, so one, one thing that, uh. You know, [01:34:00] is often used is that you create several different models and you kind of let them compete against each other to see which fits the data best.

    Um, and I'm just curious like how you think about that in relationship to this paper, because I mean, in some sense you have sometimes this using only one strategy using only other, and then using both of them together. Uh, so in some sense you do have a, a bit of using multiple models here, but I'm just curious, it seems to me that the overall approach here is still to just present one model and say, this seems to kind of work.

    Um, yeah. Did you like consider, uh, adding other models to compare it to, or how did you Yeah. Can you maybe just generally comment on that point? 

    Jesse Geerts: Yeah, so I think that sort, uh, yeah. The point of this paper is kind of, sort of a big. Work where we showed that this [01:35:00] one thing explains a whole bunch of things in the literature on a very, uh, on a very sort of like broad scale.

    Had I had all the original data sets, uh, I would definitely wanted to do that, but that's not, yeah, I, I, I think like people would take that kind of approach usually, uh, have the original data for one, for one dataset, um, and then like, analyze it really well and, uh, start with a, uh, model comparison for a bunch of different models that could explain that particular piece of data.

    This type of paper is more a sort of like ad hoc, trying to make sense of a whole different from the different parts of literature that were headed to. Unrelated. Um, for example, we had this like, yeah, the, the, the sort of blocking study that [01:36:00] was run in our lab and also this two step task for model-based model free.

    Were not designed the, the, like these tasks. Like, and, and, and those studies were not at that moment designed to be like combined with each other. And the, the, the point of this paper is more to sort of, uh, theorize and think of how these things could relate to each other. Yeah. So if like, um, if we were now, for example, to run some, uh, kind of experiment, we were, we were wonder wondering about this model or whether this model could explain our new experiment plus all, uh, uh, whether this model could explain our new experiment.

    We would definitely do that. We would run this model, fit it to our particular data and fit a few competing models as well. Is that, do, do you get what I mean? 

    Benjamin James Kuper-Smith: Yeah, yeah. Um, I think in a way my. The criticism I kind of have here seems a bit mute, but one general question is like, [01:37:00] how do you evaluate whether, uh, the model that you have then actually ex fits or explains the data you have?

    I mean, again, it seems to me that if you just look at the figures that you've presented from the original data and from your simulations, they look very similar on a like qualitative basis. But like how do you make that judgment? Like what if I don't know, you know, like some of these things aren't quite as clear.

    Like how, do you still feel confident then in saying that this explains the data? Or was it just saying like, well look, these are so obviously similar that we're happy just using that as evidence? 

    Jesse Geerts: Yeah, well, so I think a lot of the, the data. Like, so partly this is born out of a sort of like, we had to do it this way because we don't have access to the original data because like for a lot of these studies we just don't.

    But yeah, partly a lot of these [01:38:00] experiments were really quite simple with sort of like binary outcome variables. Uh, so, and, and the points that we were making were more about a general class of model that would learn in a model based first model three egocentric, allocentric combined than, for example, making very specific claims about the exact algorithmic implementation of that.

    Um, for that I would want to one, have the original data set and, and for, for a real model comparison. Yeah, I, I, I probably like want to be there at the conception of the study, make the model predictions, uh, first then gather the data and. And basically fit, fit all the models, um, uh, on it. But that, yeah, that is, uh, definitely that would be a great thing to do, but it's just not common in this sort of like this type of paper where, where it's really more, uh, meant to [01:39:00] like trigger a way of thinking about a set of a set of different experiments that seem unrelated, but might be related.

    Benjamin James Kuper-Smith: Mm-hmm. 

    Jesse Geerts: But like that of course, leaves the le leaves, leaves a lot of work to do. Um, in terms of the designing for two experiments, 

    Benjamin James Kuper-Smith: I mean, this is, I mean this, this question in particular is like one that, or also how you just responded to the question is also, I mean, this is something that I find really difficult to understand, like exactly what.

    Um, you know, as I mentioned earlier, like what exactly is the contribution of a paper and what, like, what's enough of a contribution? Um, I think, like, for example, with my supervisor, I often have discussions that I think maybe I'm, I'm trying to do far too many things at once in a paper or something where I want to have like a, uh, you know, like almost like a full blown theory with several experiments.

    Hmm. Uh, that validated [01:40:00] in numerous ways and all this kinda stuff. But like, whenever, example, listen to the way you just talked about this. Maybe sometimes, you know, there's, there's different kinds of papers and not all, every paper has to do all of those things in one go. But I, I think that's something that I'm still at the very beginning stage of learning.

    Jesse Geerts: Yeah. I, I think that's, that's a, that's a great point. If you like, if you wanna like, uh, hear, hear my personal background of that is that, um. Uh, I, I'm, I think I'm very much like you in that regard that I, I never think that anything is ready for publishing. So, uh, like with this particular paper, I, I, I figured we need to do way more and all of that stuff, and both my supervisors were like, well, there's already something, so you might as well publish it and, uh, and, and then go ahead with your next thing.

    And with my current project, I'm again, in the same situation where I think, yeah, I don't know. We've explored like a few things could do way more. And my [01:41:00] supervisors are both pushing me to write up a paper and send it out somewhere. So it's, it's, um, yeah, I, I, I think, um, you're, you'll probably have the same in like a few years time when you're.

    Supervisor will just tell you like, this is enough. And, uh, you need to, you need to set it out and stop, like thinking about, 

    Benjamin James Kuper-Smith: yeah. Yeah. It's, it is this really weird trade off because, because I mean, with me in particular, it's, I, I did something in the first year of my PhD, which started off as a kind of a first idea.

    And we also wanted just to see on a practical level how these things work. And then we pretty much had, we had like, I mean, these are three experiments, but you know, these are much less effort than doing like the electrophysiologist something. It was just three behavioral experiments. Uh, so we had those done and they were like minor variations of each other basically.

    Um, so we had like three experiments done there and then basically said like, yeah, [01:42:00] I'm ready. I felt like, okay, I'm ready to submit this because this is already pretty cool. And then I, I wrote like a first draft and. I felt like, okay, there's still something missing here. Like I don't, I haven't quite found a good way of framing the whole thing and like showing why it's important.

    Anyway, then the whole COVID thing happened. We started doing this COVID project and um, you know, I basically haven't been working on it since last February, in a way directly. Um, and so Feb, so February of 2020. And, but then in the meantime, you know, just by thinking about the project occasionally I had some new ideas.

    I realized some mistakes I made, um, some ways in which this could be improved. And so now I'm not at the stage where I'm, I have the new version, but like, I have so many ways that make this favor so much better. Like it's, it's genuinely fair to say that it's probably like twice as, it will be twice as good than [01:43:00] if I'd.

    If COVID hadn't happened, and we just submitted it last year. So I completely understand the whole, like, don't be a perfectionist, and like if you have something that's enough, just go with it. But with this paper, I feel like yeah, we would've, we would've probably rushed with something that would've been ready.

    And we've been, I mean, the, the basic idea would've been the same in a way, but it would've been tested a lot less robustly. And I think it would've been a lot less interesting also to read. So it's this, yeah. It's a really difficult trade off. 

    Jesse Geerts: Yeah, absolutely. Um, I think, yeah, when, when to decide when a project is finished is sometimes pretty arbitrary or Yeah, there's there at some point you're gonna have to make a cutoff.

    Um, I could, yeah. Uh, I, uh, with my current project, I could think of like a thousand ways to make it better on the, on the other hand, just adding more stuff is not always good. Um, so. I, I'm current. Yeah. I'm currently, [01:44:00] um, collaborating, um, on this common filter project with, um, with Sam Gershman. And, and he at some point made the point saying like, just adding more simulations or like, like trying to find more tasks that your model fits to or whatever, that doesn't necessarily make the paper better.

    Like the ba the paper is the, the paper is good if it's clear and short and concise. And if you can make your point in like short and concise, nice paper, that's probably better than having some other, uh, simulations tacked onto it just for, just for the sake of it, you know? And I think he's having 

    Benjamin James Kuper-Smith: this like thick book of simulations that Yesa made.

    Jesse Geerts: Yeah, exactly. So, so that's, yeah, I think he makes a really valid point. Um, so I, I'm trying to now yeah. Sort of take that to heart and sort of stop doing stuff and just writing, which, uh, is sometimes quite hard. 

    Benjamin James Kuper-Smith: Yeah, [01:45:00] I mean, I guess you're in the really great situation that you have. I would, Ima Well just, uh, then people also helping you out, you have a lot of experience in these kind of things, so I feel like if also if Neil says like, this paper's ready, then there's probably a fair chance that he knows roughly what he's talking about.

    Uh, same with Sam Gershman. Um, 

    Jesse Geerts: yeah, I have three people telling me that I should write to, 

    Benjamin James Kuper-Smith: yeah, exactly. Yeah. I mean, not to, I mean, my, my supervisor also knows roughly what he is doing. Um, but it's also in some sense we're starting in a field that he hasn't worked in that much either. So I think on, on that perspective, there's also just about this research area.

    A bit less experienced, but yeah. But I, I, I guess it's good to have like three people. If, if they all tell you the same thing, there's maybe a reason why. 

    Jesse Geerts: Yeah, to be fair, sort of the, like, Neil is very, um, experienced in sort of fields like spatial memory or like memory in [01:46:00] general and, uh, like spatial navigation or spatial uh, representations.

    Um, and, and this kind of stuff also, yeah, these ho field types of models of memory. Um, but reinforcement learning is not necessarily the field of our, that our lab, uh, works in. So, yeah. Uh, I'm like, like the, the lab is moving into it now a little bit more. Uh, uh, when I started my PhD, I was the only one in my lab working on reinforcement learning.

    Um, of course, uh, Sam, uh, that is his main field is, uh, reward learning, this kind stuff. Yeah. 

    Benjamin James Kuper-Smith: He knows the odd thing about reinforcement learning, decision making. 

    Jesse Geerts: Yeah. He's, he's heard of it. 

    Benjamin James Kuper-Smith: So, um, I think, uh, two. As a general point, I think you can probably wrap this up in like 20 minutes or something at most.

    Uh, but I'd like to kind of, before we [01:47:00] end it, just to speak a bit about like kind of what's, what's, what's next for Yesa other than sitting in a pandemic and trying to finish your PhD thesis. Uh, assuming that that actually happens at some point. Yeah. Do you already know what, uh, what you wanna do, what you, what you're going to do afterwards?

    Are there specific plans or? 

    Jesse Geerts: Yeah. Um, so I think just like you and probably like a lot of people, I've, I've been finding it pretty hard recently to work and, and, and prob probably even harder to, um, to make, uh, next career plans. So I have the, I'm fortunate enough to have a postdoc offer from Neil. Uh, so I can stay on for a little bit or, yeah, basically for, for at least a year or so.

    Um, I'm taking him up on that. Uh, I'm staying in the lab for a while. Um, I would like sort of after, [01:48:00]after a lot of sort of top down theory type work in my PhD, um, I think I would like to get my hands on some actual data and, uh, doing some, uh, either high level analysis, model fitting, this kinda stuff. And there's also a chance that I won't have finished the, uh, common temporal differences paper, uh, yet before the end of my PhD.

    So I'll have some time to write that up as well. And then in September, um, I'm planning to take some time off probably like, uh, I dunno what the world's gonna look like by then, but, um. If it's possible, I'd love to take some time off traveling, uh, trying to go rock climbing in some different places in the world.

    Um, um, preferably like, uh, Italy and, uh, spend some more time in the Netherlands as well, seeing my family. Um, after that, I think, so I, [01:49:00] I need to, I need to have a good think about what I want. Uh, I really enjoy doing research, so it's quite likely that I'll be doing some kind of postdoc as well. But I, um, I haven't quite decided with whom, um, in what kind of field.

    Benjamin James Kuper-Smith: Okay, cool. So yeah, just, yeah, basically finish what you're doing right now and then take some time off to think about what's gonna come next. 

    Jesse Geerts: Yeah, exactly. Yeah. And hopefully. In a COVID free world. 

    Benjamin James Kuper-Smith: I mean, by then, I'm sure we'll have the next plague. This is a, this is a nice cycle, I'm sure. I mean, like, if you think about it, we've had, uh, like if you think about since the two thousands we've had like the, what was the, the bird flu or whatever.

    I think, did we have swine flu then? Um, we had, uh, Zika, we had Ebola, uh, and now co COVID, so it's roughly every five years. There's like another thing that, [01:50:00] uh, most of them, I guess didn't actually become global pandemics, but so, you know, we're like, I guess four years away from the next one. Yeah. You, you should have like a year or two in between without, without a pandemic.