Dan Quintana is a senior researcher at the University of Olso, where his research focuses on oxytocin, autism, and meta-analyses. In this conversation, we talk about Dan's primer on synthetic datasets, science comunication, Everything Hertz, and podcasting in general.

BJKS Podcast is a podcast about neuroscience, psychology, and anything vaguely related, hosted by Benjamin James Kuper-Smith. New conversations every other Friday, available on all podcasting platforms (e.g., Spotify, Apple/Google Podcasts, etc.).

Timestamps
0:00:04: From Australia to Norway
0:09:37: Synthetic datasets
0:41:15: Software tools in science (for writing and analysing data)
0:58:41: Dan's multifaceted online presence / science communication on social media
1:06:32: How to grow on Twitter with no followers
1:15:45: The sound of your own voice
1:22:30: Some of Dan's favourite podcasts
1:25:53: How Everything Hertz grew over time
1:33:04: Finances of podcasts
1:41:45: Podcast editing

Podcast links

Website: https://geni.us/bjks-pod
Twitter: https://geni.us/bjks-pod-twt

Dan's links

Website: https://geni.us/quintana-web
Google Scholar: https://geni.us/quintana-scholar
Twitter: https://geni.us/quintana-twt

Ben's links

Website: https://geni.us/bjks-web
Google Scholar: https://geni.us/bjks-scholar
Twitter: https://geni.us/bjks-twt

References
Brown, N. J., & Heathers, J. A. (2017). The GRIM test: A simple technique detects numerous anomalies in the reporting of results in psychology. Social Psychological and Personality Science.
Heathers, J. A., Anaya, J., van der Zee, T., & Brown, N. J. (2018). Recovering data from summary statistics: Sample parameter reconstruction via iterative techniques (SPRITE). PeerJ Preprints.
Kuper-Smith, B. J., & Korn, C. (2021, Oct). Decomposed 2*2 games - a conceptual review. PsyArXiv. https://doi.org/10.31234/osf.io/5jxrf
Quintana, D. S. (2020). A synthetic dataset primer for the biobehavioural sciences to promote reproducibility and hypothesis generation. Elife, 9.

Links
The psychpathy measure used in my study with prison inmates: https://en.wikipedia.org/wiki/Psychopathy_Checklist
Most standard statistical tests are linear models blog post: https://lindeloev.github.io/tests-as-linear/
Dan's presentation on synthetic datasets for RIOT Science Club: https://www.youtube.com/watch?v=0fAR_oro1NY
Simul for writing collaborations: https://www.simuldocs.com/
Melon for live streaming: https://melonapp.com/features

Some podcasts Dan listens to:
Quantitude: https://quantitudepod.org/
Very bad wizards: https://www.verybadwizards.com/
Ologies: https://www.alieward.com/ologies
Ezra Klein: https://www.nytimes.com/column/ezra-klein-podcast
Harcore History: https://www.dancarlin.com/hardcore-history-series/

[This is an automated transcript with many errors]
Benjamin James Kuper-Smith: [00:00:00] I noticed one kind of really random coincidence was comparing the episode, which is you're from Australia and you live in Norway. Mm-hmm. You know, and I've had 30 episodes or 30 guests on my podcast so far. And you're not the first person with a Norway, Australia connection

Dan Quintana: really? Who

Benjamin James Kuper-Smith: you? So as in Hannah Watkins, um, I mean she did kind of, you know, kind of standard academic work.

Did a PhD, did a postdoc, but now she works in Australia for the Behavioral Insights unit, which I guess is like a nudge unit or kind of, you know, trying to get doctors to prescribe few antibiotics and that kinda stuff. And I think her middle name is Ard, says Hannah Magar Watkins. And uh, I think she's actually half Norwegian, half Australian or something.

I dunno, I didn't ask her, but. There can't be that many people. Like if you look at the amount of Australians and Norwegians, there are, there can't be that many. Like the, the intersection of those two circles has to,

Dan Quintana: well, [00:01:00] there's quite a few because I, I, I think a lot of the universities have these exchange agreements.

And, um, I actually, before I even knew anything about Norway, I was very close to actually doing an exchange year in my undergraduate in, in Norway. I didn't end up doing it.

Benjamin James Kuper-Smith: Oh, okay.

Dan Quintana: Um, and, um, there's a lot of Norwegians come to study in Australia. That's how I met my wife. She came to Australia to study, married her, moved back to Norway, and um, yeah, there's, there's a lot of back and forth within sort of medicine within sort of ecology in those sorts of areas.

So yeah, there, there's quite a few, um, Australian academics in Norway and, and vice versa, surprisingly. Surprisingly.

Benjamin James Kuper-Smith: So, yeah, I thought that, you know, because you basically can't be further away to countries on, on the globe basically. It's

Dan Quintana: basically the ities. Yeah. More or

Benjamin James Kuper-Smith: less. Exactly. And do you know why?

Is it just, I guess, I mean, Australia is always an, an interesting destination for exchange students

Dan Quintana: from Europe. Yeah, it's always, it's it's always seen as very exotic in Norway. Yeah. Yeah. Um, they, they love home in Norway here for some, for some reason. And, um, I, I, I think it's the same sort of thing. We see Norway, it's like a, as very [00:02:00] exotic as well.

So it's uh, it's a nice little exchange.

Benjamin James Kuper-Smith: I guess also for Norwegians, there's a thing that you can escape. Norwegian winters actually get some sunlight for half of a year.

Dan Quintana: Well, that's what they tell me. They're like, why did you, why did you move here? Yeah. I mean, you're missing, you're missing the summers.

Um, but, uh, you know, there, there's, there's much, there's much more to life than the weather. And, uh, I think the weather, the weather's fine, but the, the only hard bit is the sunlight. Where in winter

Benjamin James Kuper-Smith: Yeah.

Dan Quintana: You know, sun goes down around three o'clock, gets up at around nine. Um, but other than that, I think it's great.

Benjamin James Kuper-Smith: Yeah. No, I mean, I've, I've never been to Norway, but I spent half a year in Stockholm. Okay. And, but yeah, I mean, the summer was beautiful, but

Dan Quintana: Oh, it's amazing. Yeah.

Benjamin James Kuper-Smith: Yeah. I do wonder they also, they did mention like yeah, when you have those like four hours of daylight in the winter

Dan Quintana: stuff.

Benjamin James Kuper-Smith: Yeah. Uh, anyway, actually, one of the reasons I also bring this up is because in, I, I try to like listen to and look up as much about you as I could kind of before this interview.

I dunno where this was exactly, but on someplace you mentioned specifically that for your postdoc, you planned to move not [00:03:00] only to Norway, but to Oslo. I think it was pretty specifically the way you described it there, that that was, um, kind of, you know, often, like for example, I mean I'm bringing this also up because I am, I've got like a year and a bit left on my PhD, so kind of postdoc application, something I'm thinking about in general.

And for me it's very much a like, where do I wanna go? I don't know, you know, anyway, basically. But I'm just curious, how much harder did it make it knowing like there's one town that's not tiny, but not huge either, uh, where you wanna go and apply for stuff.

Dan Quintana: I guess it was a lot harder and there was a lot of luck involved.

The time that I was looking for a postdoc, almost out of nowhere, a position came up, which was perfect for me, and um, I was just got extremely lucky. There's just, there's nothing, there's no other way to, to, to put it. I was either gonna be happy staying in Australia. And continued to, to, to do a postdoc in Australia.

But the idea was if something pops up in Oslo, then I became ah, to move there. Yeah. And, um, and something did, and it was just, um, it was, you know, obviously I, I'd, [00:04:00] I'd, I'd worked hard to get to the point where I was, you know, at least competitive for the position, but at the same time, um, the fact that this particular position came up doing something which not that many people in the world would've, the expertise to do was just, it was, it was absolutely wild.

I still, I still look back, I look back at my career and see so many of these almost sliding door moments going. I got really lucky there. Uh, I remember getting into my PhD, at least within Australia, you generally need to be funded to do a PhD. If you get a very high mark within your, um, uh, undergraduate, uh, degree, then you'll get government funding to do, to get support to do your PhD.

Uh, otherwise you won't get accepted. I applied for this scholarship that would, that would actually give me funding. And essentially, um, I got a letter saying you didn't win. Um, but then a week later, but then, but then, then a week later, I was told the first per you were actually ranked second and the person, ah, the person that won, had pulled out.

So congratulations. Like, if that person didn't pull out, I wouldn't have done a PhD. [00:05:00] There are so many of those situations where I think, gee, I just got so lucky. And I always wonder how, how for how long is my luck gonna ride out? Um, I'll, I'll keep, I'll keep row that wave. Um, but um, yeah, getting that postdoc offer, um, was just an incredible amount of luck.

Benjamin James Kuper-Smith: Yeah. Yeah. I find this luck that, I mean, as you said, like it's, it's not like you were sitting around ring your PhD and just waiting. I like, oh, oh, there's a chance. Luckily I can take it. You know, it, it does take a lot of effort to

Dan Quintana: Oh yeah.

Benjamin James Kuper-Smith: Be able to be lucky.

Dan Quintana: Yeah.

Benjamin James Kuper-Smith: Um, but I mean, I guess for me it's kind of similar.

You know, first year of a bachelor's, I really wanted to do research and science. And then in my second year I had, I guess a second year dip and decided, uh, well basically if tuition fees in the UK hadn't been that high or I'd been from a rich family, I might have quit. Uh, but basically I couldn't afford to spend a few thousand euro and they just say after two years I go, whatever, I'll stop.

But then. My, we had like, uh, like a tutor for like 10 people each, or something like that. Like just to kind of help people out for anything he mentioned. Oh yeah, there's these like [00:06:00] summer internship kind of scholarships you can do to do some research. You earn a bit of money and if anyone's interested, let me know.

And I think I was the only one who said, I'll do it. And the main reason I did it was 'cause I wanted to earn money so I could buy camera and, and then I did it and I was like, this is amazing.

Dan Quintana: Yeah, yeah.

Benjamin James Kuper-Smith: I think I'm gonna do research.

Dan Quintana: Yeah.

Benjamin James Kuper-Smith: And the only reason is luckily they didn't ask for grades in second year because mine were, were terrible that my, I got grades in first year were terrible in second year.

So luckily they only asked for that. And then. I mean, I guess I'm only still my PhD, but that could have ended up very, very quickly. Yeah. If that hadn't happened. Oh, and also, like there was only, there were like three scholarship things we could apply for, we thought, but one was, what's the nice word, discontinued that year.

And um, and there was one where the deadline was tomorrow. Oh, wow. And so I like literally had to drive across London to like, hand them the application of paper. But yeah. Okay. So like, it wasn't, okay, so like, yeah, like I guess you moved to Oslo was [00:07:00] somewhat specific and run out on some luck, but it wasn't as, um, from the way I remembered, it was really like, I'm gonna move to Oslo and do, and then

Dan Quintana: well look that, that, that was what I really, really wanted to happen.

Um, I could have probably found something within Australia or within Sydney. There's a couple of institutions within Sydney that would've been fine. But the intention was I really, really wanted to go overseas and, um, you know, obviously. Norway made a lot of sense because my wife's family lives around Oslo and I thought, I wanna make this happen.

And I, I was really, um, boxing myself in because there really is. Only one institution within Oslo that does my sort of stuff. I mean, there is a few other institutions that do other stuff that I maybe could have been applicable for or maybe could have been more suited for. Um, but, um, yeah, look, I just, I just rolled the dice and the op, the op, the opportunity came, which was, um, which would, which is incredible.

And I still pinch myself honestly.

Benjamin James Kuper-Smith: Yeah, so, so it seems like Norway was also then. I mean, there was the connection through your wife, but it wasn't, that seems also like coincidence about Norway because I guess you just said he wanted to go [00:08:00] overseas. I mean, you would've made your life easier if you said, I I'll go to London, for example.

Right. As

Dan Quintana: an Aussie. As an Aussie, of course. Yeah. But I mean, look, I've, I've always been fascinated, fascinated by Norway,

Benjamin James Kuper-Smith: I mean also in terms of just the amount of labs that are there.

Dan Quintana: Oh, yeah.

Benjamin James Kuper-Smith: Just the opportunities.

Dan Quintana: Much, much, much more opportunity.

Benjamin James Kuper-Smith: You wouldn't really have to have been lucky, even

Dan Quintana: get something.

I mean, there would've been, yeah, there would've been heaps of opportunities. But look, I, I've, I've just, um, I've always been fascinated by Norway. Um, before, before I moved there, I'd visited a number of times, um, with my wife. Um, so I thought, this is, this is, uh, this is a, this is a really nice country. And, um, do not regret a moment of doing it.

I mean, of course it's hard being away from family who, who are back, family and friends back in Australia. But, um, everything else, it's, it's, it's fantastic here.

Benjamin James Kuper-Smith: That's cool. Yeah, I mean, I've, I think I've only heard good stuff about Norway from people who move there, who know, it's maybe, maybe one day I'll see a position

Dan Quintana: apply for it.

The, the beer is too expensive. That's my only, yeah, that's my, that's my only complaint. I have to, when someone goes, oh, drunk that for some beers, I have to check the bank again. Sorry,

Benjamin James Kuper-Smith: I'm, I'm only a research like

Dan Quintana: senior

Benjamin James Kuper-Smith: scientist. I'm not a professor

Dan Quintana: yet. I'm not a professor. I, I [00:09:00] can't, I can't take, I can't take the hit or you have to, they have this thing, um, it's called, um, the, the, the literal translation is like salary beers.

So like the Friday after salary usually gets paid. Everyone goes out for drinks because that's, that's the one time of the month that everyone can afford it. Exactly. It's, um, that, uh, yeah, it's,

Benjamin James Kuper-Smith: I mean, I'm sure if you, you know, work hard and save your minor, you can afford to be it even on a different day.

Dan Quintana: Oh, I know. If

Benjamin James Kuper-Smith: you really put your mind to

it.

Dan Quintana: If, if you, if you really want it.

Benjamin James Kuper-Smith: Yeah. Yeah. Um, anyway, so I think this episode is gonna be a, a one of very rough transitions from on topic the next, because synthetic data sets and podcast, there's basic no link. Uh, so. So I'd like to talk about the basically synthetic data sets and your prime in particular, because Yeah, I think it's really interesting and I hadn't, I don't think I've heard about this concept outside of, I've probably heard about this on your podcast.

I'm assuming you've mentioned it and everything hurts at some point. I think so,

Dan Quintana: yeah.

Benjamin James Kuper-Smith: And I don't think I would've heard about this other than that. And maybe if I randomly come across your primer, maybe [00:10:00] could you maybe for a minute or two kind of roughly describe what this is, what the, what synthetic data sets are, why they're interesting and, uh, yeah, just as a kind of, kind of get the ball rolling.

Dan Quintana: When it comes to synthetic data sets, this was a case of me trying to scratch my own, my own itch. In terms of the problem of, I wanna share my data, but a lot of my data is very sensitive. You don't wanna disclose a lot of this information for some of the populations that I'm working with. Um, I do a lot of work with autism, for instance, but there isn't that many people with autism within the Oslo area because there's only about a million people in Oslo.

So it's very difficult to keep or maintain. Um, you know, participants wanna keep their privacy in, in, in a lot of circumstances. So I think when a lot of people are talking about sharing data, they speak of it as if it's a very easy task. They're like, why don't you just share your data? Because, and when, when you drill down to it, a lot of these people are working with like reaction time data, for instance.

[00:11:00] Um, yeah, or, or visual perception in which sharing visual perception, reaction time, data, um, there, there really isn't any risks when it comes to disclosing that sort of data with, with your participants and. O obviously, you know, within the bi behavioral sciences, we're experiencing reproducibility crisis, and one of those ways that we can improve or increase the robustness of our results is by sharing our data sets.

By sharing your data sets, um, other people can verify your work. Um, that that's including peer reviewers, but also other people. But these open data sets can also be used to generate new hypotheses. As, as we know, collecting data can be expensive, very time consuming. Um, but if someone has already collected data set, which is addressing the sort of research questions that you're interested in, and the data's there and it's open, you can actually go in there and you can do some hypothesis generation and you can analyze the data and you can say, this really cool, this cool, this really cool thing popped up.

And then you can use that as a basis for, uh, future [00:12:00] hypothesis driven research. So there's a lot of benefits to open data, um, verification, but also generating new hypotheses. But we have these challenges within a lot of populations, particularly if you're working with clinical populations or vulnerable populations.

And you have this tension between the utility of open data and also the need for disclosure protection. Open data has no disclosure protection, um, but no utility. Um, sorry, but plenty of utility, but closed data has a lot of, um, disclosure protection, but almost no utility. So synthetic data sets bridge those two different things because essentially with this synthetic data set, what you've got is you've got a data set which, um, mimics the statistical properties of your original data set.

Yet no participant in your synthetic dataset represents a real individual. So you can go through, say, say you have, you know, a hundred individuals and each of them you have 10 variables. It could be age, it could be some score on some sort of personality test or something. Um, a whole bunch of demographics.

Um, if you go through every single [00:13:00] one of those participants with the synthetic data set, not one of those rows will represent a real person. And so what you can do is with that is if you are publishing a paper, then you can say outright. Um, we couldn't share our data, our raw data, um, because of, um, of, of, of, because of privacy, which I think is reasonable in many circumstances.

Yeah. But we've shared a synthetic data set which mimics the same statistical properties. You can run the analysis and you will get more or less the same. You, you'll get the same conclusions and more or less the same numbers. And as a benefit, um, you can actually do some exploratory analysis on this data as well.

And this is a very handy way, and I think it, it solves this problem of how can we share our data while protecting participant privacy? And this was something I was thinking about for my own research because of the vulnerable, vulnerable populations that I work with. And I just started doing some digging around, and I discovered this because this is actually commonly done with census data.

We're talking about like [00:14:00] hundreds of thousands of people. And the government obviously, obviously wants to share this data so other researchers can, can ask interesting questions in terms in terms of demographics. But of course you need to respect the privacy of participants and or people who have, who have taken the census.

So this. Idea of synthetic data was originally came about in order to solve that problem. But only more recently has it been applied to, um, to the Biobehavioral sciences or to to, to medicine. And I'd only seen like two studies which have actually used this. And I thought, I need to promote this. This is super interesting.

And when it comes to, I, like I, every year I give myself a project. I give myself a meta science project. It's my summer meta science project. I'm a nerd. That's what I do for fun. And that particular year, I'm like, I'm going to learn this thing back to front, to the point where I can write a tutorial paper on it so that I can use this for my future work.

Um, but also so I can promote this to other people. And I always think when it comes to writing papers is that if you are helping other people scratch their rich, making them feel like the hero that like, this is super [00:15:00]easy, I can do this myself. Those are the papers which get a lot of attention and, uh, that, that, that are the most useful.

And it's, it's amazing. Like with, with so much of this, um. Sort of meta science stuff. This is the stuff that, you know, granting agencies. And when it comes to promotions, it doesn't matter as much. They don't really care about this sort of meta science stuff. But when it comes to the stuff that's having the real impact, getting the most attention, the most clicks, this is the stuff that actually gets the most, the most attention there.

So I, I wrote this, um, I wrote this paper for how to actually create your own synthetic data sets. And I think another benefit of doing this as well is that sometimes when you're reading a paper, it can be very difficult to follow the analysis of what people have done unless they're doing something very basic.

We ran some t tests. I'm like, okay, that's fine. But some people have some very complicated analysis and as much as they try and describe it, it's still very difficult to follow what they're doing. Some people may actually add an R script, which can help to see what they're doing. But still, the r script can be difficult to read.

But if you are sharing a synthetic data set, you can literally run the [00:16:00] whole analysis with this script that they're sharing and so you can better understand what they're doing. So when it comes to this analysis that, that, there are two things that you need to consider. It's not a matter of just pressing a button and it's automatically gonna generate the synthetic dataset in, in some cases it is actually that easy, but you need to verify that it does represent the original data.

And so there are two things you need to consider, whether it has, um, general utility that is, does the data behave the same way in terms of what are the means and standard deviations of the variables? What are the relations relationships between variables? You wanna make sure that those things are behaving roughly similar.

Benjamin James Kuper-Smith: And that's like a thing you have to test manually.

Dan Quintana: Exactly. Yeah. So this, this, this is part of the synth pop package, which is what the tutorial is about. So when you're running it, you wanna test, does it have good general utility? Do these data sets roughly behave the same? The next thing is testing specific utility.

In that, do the analysis that I run in the original dataset, mimic the synthetic dataset, and you can actually report, there are [00:17:00] statistics that you can report so you can actually say, yep, these things are essentially the same or, or very similar. And once you have those two things, then you can be very, very confident that one, the data, the synthetic data set can reproduce the same results as the original data set.

And if you have strong general utility, that's also a really good thing for data exploration because you can be fairly confident that if someone else is. Um, having, having a play around with your data, they're gonna get roughly the same conclusions as if they're playing with the real data. But the benefit here is if someone does that, then they can say, Hey, I played with synthetic data set, I've got this result.

Can you run this script on the real data just to verify that it's the same thing? And it's, and it makes it a lot easier to actually do that sort of exploratory stuff. So yeah, there's a ton of benefits here and um, it's really encouraging to see a lot of people beginning to do this within their, within their papers.

I set up my Google Scholar alerts for when people are citing my stuff, and one of the most exciting alerts is to get someone who's used this in their own research. 'cause I'm like, great. It's one more paper where someone's, um, data can [00:18:00] be, can be verified and someone can actually share their data so other can, other people can run, um, exploratory analysis in the data.

Benjamin James Kuper-Smith: I guess. It's interesting, right? Like every time you get an alert for that one, that means there's an open date just head that probably wouldn't have been open before.

Dan Quintana: Yeah. I, I love

Benjamin James Kuper-Smith: that. Yeah. I basically like to, you know, talk more about all the different strands you kind of mentioned in that, um, introduction.

Yeah. I find it like maybe to, to, with something you mentioned at the beginning, like, I, I agree, like sharing data always sounds very easy and then even if you have a basic data set, I mean, in a sense it is, should be relatively easy. But I've noticed since I've started, I basically, I started with the intention of sharing data and code from the beginning of my PhD, but only once we came to the stage where we wanted to publish stuff did I realize that my code and data probably wasn't in a necessarily published format, and then I had to, you know, write it better so it can be improved and, um, uh, so it can be published.

And I, yeah, I think that's, it's an important point to [00:19:00] say like, sharing data isn't the easiest thing. Even if you have data that is in principle. Parable, like, you know, it doesn't have these clinical things. Um, yeah, I think one thing that I, uh, thought we could do is kind of use one study that I'm thinking about applying this to, and then kind of using that to kind of see what some problems might be with it.

Yeah. Or where it works or doesn't work and that kind of stuff. So the study I'm for, for which I basically read your article, I do fairly like basic decision making, economic games and social interactions kind of stuff. Not really the data where there's really any problems about privacy or anything like that.

It's an anonymous data of course, but there's no real problem there, per se. But then we randomly kind of had the chance to collect data with prison inmates who were in a, uh, I mean, so this was, uh, in or around Hamburg where I started my PhD. Our lab kind of moved halfway through and yeah, so we got the chance to, to collect data in a.

I dunno whether it's [00:20:00] maximum or high security or whatever exactly the levels are. But some of the, some of the people who are there and who I've met are some, some very bad boys and they've done some very bad things. Um, I think including murder and child rape and all sorts of things. And the interesting measure here for us is this psychopathy measure that PCLR score, um, which every, um, I think every image gets that.

The problem is if you have a super high psychopathy score, it means you're, you know, basically the definition is you're a psychopath beyond a certain point. And basically if you know this person is this age and has probably done something heinous in or around handbook, you can probably more or less read than like Google the news and figure out who it is, even in a city as big as handbook.

So. That's the kind of general thing where the, the data I have is pretty straightforward. It's just some binary decisions, corporate defect, some scales from zero to a hundred. That kinda stuff like it's a fairly basic, uh, data set. But where, yeah, where I couldn't really share the original data set, um, because I'd be afraid that some people would [00:21:00] be identifiable where name, uh, with just a quick Google search.

Yeah. So that's kind of the, the, the, the rough description. Maybe one quick question. I mean, you, you mentioned that this was developed for, uh, sensor data and that kinda stuff where you have enormous data sets and that kinda stuff. Does it make sense to do this also if you have 40 participants? I dunno exactly like how the, you know, how the inside of the, like kind of exactly what the algorithms are of what this, uh, package chooses.

Um, but I'm just curious, is this something where sample size, there must be some sort of restriction, right? Like some low or upper li Well, more lower than upper limit, but

Dan Quintana: I'm not sure. Um, I've did some simulations. Testing, um, different sorts of sample sizes, and I don't re, I don't remember what my lower limit was, but even, even, even something with like 20 participants, it'll work fine.

Um, the only limitation that I can think of in your scenario is if there is a single outlier. So for instance, let's say you are working with the population and you, one person had a very, very high psychopathy [00:22:00] score. Um, and then if there is a single outlier, then, then it's, it's possible to identify that individual.

Um, although I would imagine within your sample, a number of your participants would have very, very high psychopathy scores,

Benjamin James Kuper-Smith: actually. So the, the slightly surprising thing is, and I guess somewhat disappointing after we collect the data, is that the psychopathy scores aren't actually that high. Okay. Um, they are pretty normally distributed first.

Mm-hmm. Um, so there isn't any crazy outlier. I mean, it's definitely more psychopathy than like the average population, but, uh, yeah, we don't have like a. You know, whatever. Who are the famous psychopaths? I can't think of one right now. I'm blanking. But yeah, we dunno. Someone who did something completely out in insane.

Um, so that should be too much a problem. But for example, our dataset is fairly small, so I think the entire dataset is something like 38 people. Okay. And 24 of them, I think did all tasks.

Dan Quintana: Okay.

Benjamin James Kuper-Smith: Um, but that shouldn't be a problem then, or,

Dan Quintana: yeah, so I think, so I, I've, I've tested it with samples like that, like with [00:23:00] 40 participants.

And the, the two main things you have to watch out for is if there's a single outlier, um, that makes, and

Benjamin James Kuper-Smith: so single outlier means in. Any relevant dimension or just the ones identifiable or,

Dan Quintana: uh, ones identifiable. So age is one as well. So say if you are working with a, a population of, uh, people, car, car accidents, for instance, and there's one person who, who's age 90 and you're working in a small town and you know that 90-year-old that was in the car accident.

Um, the, the other thing to also consider is sometimes when you have, are running the synthetic data sets by chance, it'll happen to reproduce the same values as the real values. Um, this is more likely to happen when you have massive data sets. So if you have like a thousand participants, if you have a thousand participants and you have five variables that are limited age, some psychopathy score, and um, the highest degree of education for instance, you are bound to have, it's just [00:24:00] impossible.

You can't create a synthetic dataset, which isn't gonna have somebody that's H 30 that has a high psychopathy score and who has a high school education. So in that case, although the data is scrambled, what's been recommended is to actually leave those people out. Because if somebody actually, even though the, even though it's very difficult to identify them, if they were to reread your study and people who are interested in research tend to do that, then they might look in the data going, hang on, that's me.

And, and but you, you, you promised me I wouldn't be identified. Yeah. So in those cases, it's, it's, it's possible. It, it really depends on the nature of the data that's whether that's gonna happen. But as I mentioned in the primer, it's very important to run those diagnostics to actually see, has any of the same data been matched?

So does the synthetic data match the real data? What you can do is you can almost do it, it's kind of like synthetic, so it's, it's like p hacking of the synthetic data. You can set different, you can, you can set different seeds

Benjamin James Kuper-Smith: for like randomization or,

Dan Quintana: yeah. [00:25:00] And sometimes if you change your seeds around, then you can avoid this problem.

Other times when you act, when you're doing the synthetic data analysis. What can change how it's generated is what you set as your first variable. So sometimes if you set, if you change around what the first variable you put into the model, then it will change how many people are actually matched. So there's a few different things that you can do, but doing this as well as actually determining the general utility and the specific utility, it's also very important to do these additional diagnostics in which you actually check one, is there any big identifiable outliers?

One thing that you can do with that is you can potentially do some binning. So rather, rather than treating, um, age as a continuous, um, variable, then you can sort of say everyone above 80, for instance, not ideal, but if you want to keep everyone in, then, then you can do that.

Benjamin James Kuper-Smith: I guess it's also maybe important to, to just remember that this is about the data set you share, not about the analysis you do.

Right. You can, it's, you know, the bending here is only for sharing the data rather than Exactly. [00:26:00]Rather than for actually running analysis. Yeah.

Dan Quintana: Uh, I think other consideration as well is a lot of people think, oh, I don't wanna share my entire data set. That would be ideal, but I think as a bare minimum, if you can share the data set that's required in order to actually reproduce the analysis, you, you may have collected 20 variables, but only reporting five.

If you're assuring the five, that's enough. If you wanna share the 20, that's also fine. Um, but yeah, those are the main considerations that you wanna take, that, that, that you wanna take care of. How's the general utility? How's the specific utility and am I maintaining the, um, the, the privacy of the participants?

Um, so if you're working with a large data set, you know, typically with a large data set, you might see 1% of participants, like I'm talking like 2000 people of that maybe like one to 2%. There's just no way around it. They're gonna be the same val values. Then one recommendation would be just to remove those participants.

If by removing 1% of your data, you, you, you are changing the outcomes of your results, then I wouldn't, I wouldn't say your data's very robust to begin with. Yeah,

Benjamin James Kuper-Smith: exactly.

Dan Quintana: And so it's very [00:27:00] important to be upfront going, Hey, here's a synthetic data. Uh, 1% were removed because they were matched case wise. Um, however, the re the results are the same.

So it's, it is very important to emphasize that you're not aiming to get exactly the same. Numbers. It's just, it's just not possible. But that's not the point. The point is, can you reproduce the overall analysis and can people get roughly the same outcomes? And, um, if it means you removing some participants for the sake of, for the sake of participant privacy, then I would say, then I would say, go for it.

But look, it's, it's, it's hard to, this is just speculation sometimes, like with the data sets that I'd used in the primer, these ones didn't have any matched, I don't think from memory. But it all depends on the type of data that you have. Um, if you have like reaction time, data, you know where, where things are from, anywhere from sort of like 200 to a thousand milliseconds, the chances of you having exactly the same thing, they're gonna be very slim.

But if you're working with some types of variables, then you're gonna get matching. So these are some, these, these are some considerations for when you're actually working with synthetic data sets.

Benjamin James Kuper-Smith: Yeah, I guess [00:28:00] in relation to mine and then probably also other people's data, I guess some identifiable variables like age probably are at least, potentially aren't actually that interesting for your analysis.

So for example, in my case, I don't really care about age actually, so I could just have the psychopathy and their behavioral things and then, okay, there's someone with a psychopathy score in and around Hamburg. Then it's, yeah,

Dan Quintana: that, that then you're fine. Like may, maybe you're reporting it in as your demographics.

Um, sometimes it might be relevant to include age within a model. I don't know. But for a lot of, in a lot of circumstances you, you don't need to include it. So, so don't,

Benjamin James Kuper-Smith: or I don't need to include it in the shareable

Dan Quintana: Exactly.

Benjamin James Kuper-Smith: The shared data set. Exactly. Yeah, I guess. Okay, that's a good point. Yeah, I never thought about that.

Um, okay. But I have one. So in a way I really like the idea and it sounds really cool, but somehow I'm not entirely convinced it works. Maybe this is just because I haven't looked exactly under the hood of how it works or whatever, but I probably have to ramble a bit to make this question. But I hope I'll make my point.

And that is basically my question is kind of how. Close the relationship is between your [00:29:00] actual data and the synthetic data you create. Because let's say I have 20 variables or something, right? And I want the relationship between all of them to be maintained. It seems to me that beyond a certain, uh, like level of detail and specificity, there will be only one dataset that will match it to that level of detail.

Like if you really want to have, you know, I want the correlations between these variables or whatever to be like, you know, until like five, uh, digit, you know, whatever, like at some point, only one combination of numbers is gonna fit that, and that's gonna be your original dataset. Um, I guess maybe the, the general logic I'm thinking about is, you might have heard of this Grim and Sprite, where you know, you, you say like, okay, these numbers can only be created by this kind of combination of values.

So I'm just curious isn't. It just seems to me that if you have a large amount of variables, that at some point, unless you are pretty vague in terms of how well, how [00:30:00] closely you want them to match that at some point. Yeah, it will just, it will, I'll say the more precise you want, the relationships to match you will converge on your actual data set.

Dan Quintana: Yeah, exactly. And this is why you, you always have to check this, this is one of those checks that you do within this check of general utility. Firstly, you're looking at the frequencies of you, you're, you're sort of comparing, you know, how many males are there in the original and how many males and females are there in the synthetic.

But you're also, um, one other thing you do is you would run Scatterplots to actually look at the relationship between, um, um, the,

Benjamin James Kuper-Smith: uh, okay,

Dan Quintana: so, so you do catapults and like, uh, where it may fall down is if you're looking sort of at, at multi-level stuff. For instance.

Benjamin James Kuper-Smith: Yeah. For just anything more than like a correlation or something.

Yeah,

Dan Quintana: yeah. But, but, but typically what you would do is within, within these examples, you can run, um, you, you can fit a linear model with, um, a range of coefficients and including interactions. [00:31:00] In that sense, you can actually see how closely did the coefficients and, um, the variance of those coefficients match between the original and the synthetic data set.

And in, in many examples, they match very closely. So that sort of demonstrates that, um, within the synthetic data sets still do maintain the relationships between variables. So it's one thing to actually say, okay, the means and the standard deviations for all the variables are very similar, but what it also does is it maintains the relationships between the variables.

Now it's important to actually check that this actually occurred, especially for your analyses of interest, which is why you do these tests of specific utility. Now, one thing for these particular tests to work, you need to fit it as a linear model. Um, one thing which blew my mind, which I can't believe I wasn't taught during undergrad, is that every single common statistical test is essentially a linear model.

So if you are, uh, if you're not, if you are not already using linear models, you may need to convert your T-test, for instance, into a linear model. And then. [00:32:00] Within the synth pot package, then you can basically do those comparisons. You fit a linear model and you can compare the linear model between the original dataset and the synthetic dataset.

So what you're doing is you're just seeing how closely do the coefficients and the variance of the coefficients match up. And, um, it runs a statistical test to actually show you how similar these things are. And you can even report the percentage. So there's, like, you, you, you might say there's a 95% or there's, there's a 93% overlap between the synthetic coefficient and the original coefficient.

So I, I think one really important thing to stress is that sometimes it doesn't work just through the nature of the data. Mm-hmm. Um, but I don't think it's necessarily, necessarily a bad thing. You can be very upfront in your paper going, we attempted to, to, to create synthetic data. Um, the, the, the, the results differ a little bit.

However, we're including this in order to re, in order for interest people to reproduce the analysis so someone can actually go through and, and, and rerun the analysis. You might get different results, but at least people will have a better understanding of what you actually did for your [00:33:00] analysis. So these are important things to check.

Um, it's not, it's not gonna work for all sorts of data, but it, it works for a lot of types of data.

Benjamin James Kuper-Smith: Um, a few points related to what you just said is like, one is that it seems to me also that just reproducing the summary statistics of your variables is. To honest where, where that went, if you just, if it does, just did that.

I didn't quite see what the point is because I thought like the main kind of advantage is that other people can do exploratory analysis on your data and that really requires the relationship between variables to be intact, right? That's kind of the crucial thing there. Well, I,

Dan Quintana: I still think it can be interesting because that's one way of actually checking, like, is this data weird, for instance?

Um, by looking at that and just even, even by looking at the distribution of the data, you know, for, for instance, you might be very familiar with the type of measure and you know that this measure always gives you a few outliers, but if someone reports dataset, okay, okay, with no outliers, you can go, that's a little bit strange.

Um, so it just gives you, even looking at the distribution of the data that the means, the standard [00:34:00]deviations and the presence of outliers just gives you a better idea without necessarily even running any analysis. I still think that's better than nothing.

Benjamin James Kuper-Smith: Yeah, that's a fair point. I mean, I guess this also relates to, um, where I, I talked to Joe Hilard about his, uh, his Zang affair and where he got data that was just a cube, like the th just really weird looking data where based on that, you basically say like, no one collected this data.

This isn't,

Dan Quintana: yeah. And, and when you see the synthetic data, because it, because it roughly approximates the real data, you can still see this is weird, this is weird. Data does not behave this way. So it's just e even, even for that, it can be quite useful.

Benjamin James Kuper-Smith: Yeah. Uh, one thing that probably should have mentioned at the beginning of our conversation, uh, I put links and, uh, I put like references in the description and links to everything we talk about.

And you mentioned this, you alluded to a blog post with, uh, linear models. I put that in there too in case people were wondering what that's about. Oh,

Dan Quintana: it's, it's, uh, oh. It's, it's, it's such a good blog post.

Benjamin James Kuper-Smith: Yeah. I haven't read it yet. I just, I saw it yesterday. I [00:35:00] watched your. Uh, what's it called? Like riot science or something like that?

Ah,

Dan Quintana: yep, yep, yep.

Benjamin James Kuper-Smith: Um, there's something I could talk on synthetic data sets by you on there, and you mentioned that, and then I had to look it up and I haven't read it yet.

Dan Quintana: It's really good. Uh, I think the, the other thing I want mention is that, um, when it comes to sharing data and sharing analyses, things like RR is fantastic because it's accessible and anyone can, it's very easy to share scripts, but packages and r versions change over time.

So something that you share now, it's, it, it's a little bit depressing to think about, but something that you share now may be very difficult to do in 10 years time because all these things update. One thing I did with this paper was I shared a reproducible docker image of my analysis, which basically docker

Benjamin James Kuper-Smith: image is what?

Dan Quintana: Yeah, so this is basically like, it's like a container, like it's a self-contained. Um, I'm probably explaining this very poorly. It's a self-contained file which allows you to rerun the analysis based on the environment, which I saved it in. So when I [00:36:00] wrote the paper, it was a particular R version and all the packages I used were a particular version.

I put together this pack, this, this sort of self-contained bit of code, which means that in your browser you can rerun the analysis and in 10 years time it's gonna behave exactly the same. So this is hosted on, on my GitHub. eLife also saves a copy in case my GitHub explodes, for instance. And then what it means is you click on this link takes about sort of, you know, maybe a minute to load and it loads up our studio, uh, our studio server on the web.

And you can rerun it exactly how I did it on my computer. And it's gonna, it's, it, it's, it's future proof essentially. And it's, it's a really nice thing that I'm looking at doing for, for, for more of my papers so that, you know, on top of sharing your code, which I think is important, um, it helps future proof your code as well.

And then this pops up in the browser. You can see it, it recreates my, um, environment. You can see all the packages that I use, all the files that are there, and you can rerun all the analysis that's explained there.

Benjamin James Kuper-Smith: Yeah. Can we [00:37:00] maybe just a minute about that topic more, because this is something I've heard about before, but I don't really, I mean to prefer, I don't use R, so maybe that's maybe the first reason why, um, like if I use matlab, maybe that's pretty straightforward.

Dan Quintana: You can, you can do the same. I think if you do it in Octave, you can do something similar.

Benjamin James Kuper-Smith: But I guess I'm just curious, like for people who are using R, what are some maybe like educational resources or you can use, or what are some good tools you use for doing that? Yeah, for this kind of archiving of the actual R version you're using.

Dan Quintana: I used a package called Binder. Oh,

Benjamin James Kuper-Smith: that's what binder's for. Okay.

Dan Quintana: Yeah. No. Is it called Binder

Benjamin James Kuper-Smith: or, I've heard the name before, but I, you know, 'cause I don't use, ah, I don't really

Dan Quintana: hear that. I have to have to look back at that. Was it Binder? No, I used a whole punch. My bad. That's the package. So a whole punch.

It basically does everything in the background for you. Um, you have to, you know, it doesn't take very long. It's only a few lines of script and it will do all the work for you and help you create this self-contained [00:38:00]package. And that's what, that's what I did for this particular paper. It's not hard, it's not easy.

And it took me a little bit of fiddling around, but, uh, look, I'm, yeah, like an afternoon sort of thing. Get, getting it all together and getting it ready. Um, but once it worked, it worked beautifully. Um, so whole punch is the name

Benjamin James Kuper-Smith: and then what you get out of this like a, a zip file that you can upload to GitHub or what exactly is the kind of date you get out?

Dan Quintana: Yeah. So how

Benjamin James Kuper-Smith: do you,

Dan Quintana: so more or less the, these, these files live on, on GitHub and then it's sort of run? Uh, I'm not, I'm not entirely sure how, how, how it works to be honest. 'cause it, it was all, it was all very new to me, but the files, the files live on GitHub and then you, you sort of run it from there. Um, but yeah, it's, it's, it's run within, within, within binder.

Um, and then you can, yeah, you can see all your stuff there, but you can do this with r you can do this with Python. Um, you can do this with Octave as, as far as I know as well. So it's quite, it's quite flexible. It's code, yeah. Language agnostic.

Benjamin James Kuper-Smith: Yeah. I might have to do this with Python soon. Uh, but actually whilst we're talking about pregnant [00:39:00] language.

One of the reasons I am hesitant to use the synth pop package is because I don't really use RI have used R before. Mm-hmm. But basically now all my analysis in MATLAB and in the future, especially for online data collection, I'll using Python and have to learn some JavaScript. I hate programming. It's the least favorite part of my job.

I'm not really looking forward to learning art, so I can use this one package or make some nice figures.

Dan Quintana: This is very straightforward compared to other packages. Um, if you're doing sort of basic synth pop, it's literally a few lines of code. So compared to other packages, it's very straightforward to use.

Benjamin James Kuper-Smith: So I have no excuse.

Dan Quintana: No excuse. And the, the, the, the, the documentation, um, the documentation is, um, is, is, is, is very good. Um, and there's a lot of support online. But, um, yeah, compared to other packages, it's very nicely written and it's relatively straightforward to use. So I would think that, um, yeah, even someone who's sort of got a basic familiarity with r [00:40:00] will be able to use it.

No problems.

Benjamin James Kuper-Smith: Ah, damnit, no excuse, excuse, no excuse, excuse. Yeah, it's, but this is, you know, this is one of those things with where the, with, you know, I'm all for open science and reproducibility and all this kind of stuff, but sometimes like, ah, another thing to do,

Dan Quintana: but I, I think as well, um, something that a lot of people forget is if you're doing.

Relatively straightforward analysis. There are tools like JAS and Jamo in which you can do exactly the same thing in which you can share your, um, you can run your analysis and share your scripts and, um, you can share your JAS and Jamo file and anyone can open that up and reproduce what you've done. Um, but that's, you know, that's assuming you're doing

Benjamin James Kuper-Smith: not for critical data, or not critical, but what's the word?

Confidential data?

Dan Quintana: Uh, no, exactly. So if, if you are, if you're willing to share the data that is, then, then you can use that. But yeah, if you wanna use synth pop, then, um, I, I actually toyed around with the idea of, of, of, of writing a, a shiny app for, um, for synth pop. Then I realize that there's potential privacy concerns because the shiny app has to be hosted somewhere and people are [00:41:00] putting their confidential data into the shiny app.

Right, right, right. So I was, I was about halfway through making it, and I'm like, oh, crap, that's a, that's a really bad problem. And I, I didn't, I didn't know how to solve that, so I sort of abandoned it. Um, but using the package within r um, if you're a beginner, you can do.

Benjamin James Kuper-Smith: I mean, to be fair, I mean there's, I guess there's like a.

A question that's always tricky, like which programming language to learn, because then it seems like R can do a lot of things that other things can't do. I mean, one thing that to me sounds very interesting is this whole idea of markdown where it automatically, like, you know, you're write, as far as I understand, you write your paper in R or whatever, and then it automatically puts the results of your variables in.

So if you, you know, end up changing something about it, then the, it automatically changes the, the, you know, slightly changed output. That does sound really cool. And that alone is for me, almost worth considering doing my analysis. Analysis.

Dan Quintana: It's, it's very cool. I've, I've played Ram with it and, um, it's, it's like magic.

'cause like, invariably you are gonna, you know, review was gonna suggest additional analysis. Um, you might want a different analysis [00:42:00] and it reduces the chances of you making an error because sometimes you're running your analysis and you're copy and pasting between your Word document or between your L Tech document.

Yep. And, um, you know, just, it's just human error. But doing it this way, it's, um, yeah, it's, it's cool. It's very cool. I've played, I haven't actually, um, maybe I've done one paper like this, but it's, um, yeah. Oh, so

Benjamin James Kuper-Smith: you don't actually, I, I assumed you would just use this all the time.

Dan Quintana: No, look, look, I don't know, like I'm, when it comes to writing stuff.

I find it almost distracting because you have your R Markdown and then you have, and then just, I, I'm always, I write a sentence, I'm like, okay, I wanna see how that renders, press the button. It takes like three seconds to render and it just gets, it gets in the way of my writing. Uh, and also my collaborators.

A lot of my collaborators don't use R or they don't use T Tech or, or whatever.

Benjamin James Kuper-Smith: Can't you like export it as

Dan Quintana: a Uh, yeah, but it's, it's, it's a pain. So you, you can export it into Word it, it doesn't always work, but then it seems like a pain to ride it, [00:43:00] export it, get your collaborators to make the changes, and then import it back in again.

Yeah. So one thing that I have been using. Um, for doing this is this service called sim, L-S-I-M-U-L. It's like GitHub for your word documents and it's super easy for you to actually share your Word documents. So you upload your document to sim to, to, to, to simil. I write it and then I invite my collaborators and then they, they can access it.

Um, and what you can do is people can work on it in real time and then you can actually version control everything and then say you have three people working on it. Once they're finished working on it, you can then merge. You can then merge those documents together. But the good thing is it doesn't actually create more work for your collaborators.

'cause all they're doing is opening the Word document, working on it as normal, and then saving and that's it. That's what I found has worked the best when working with collaborators. Sometimes if it's just a paper with me and my PhD student, we might just write it in Overleaf, for instance. Other, other times we just do, wouldn't work.

It just depends on, on who I'm working with. But, um, when it comes to [00:44:00] these bigger papers, when I've got like, you know, 15 coauthors for instance.

Benjamin James Kuper-Smith: Okay, yeah,

Dan Quintana: yeah. Like just, ah, like I, I think some people are just like, it's very funny seeing people like, you know, raise their noses. Oh, you use Word? Oh, I'm like, mate, like, you obviously don't collaborate with people.

Many, many, many people who, because a lot of people just, just dunno how to use these other tools. And sometimes I feel, um, like, oh, I, I, I was working with the Lite document yesterday and I just wanted to move a box to the bottom of the page and I was like 40 minutes in Googling and I'm like, oh, stuff this, like, if this is word you, you just move it.

So look, there, there, there are, there, there are pros and cons, but I think, I think some people are very, sort of very, um, you know, like, like software hipsters when it comes to just to science software. It's, it can be very frustrating.

Benjamin James Kuper-Smith: Yeah. No, I mean that's, I, I mean I just literally. Last week, um, we published a Preprint that had some mathematics in it, so I needed to use Overleaf or I don't know, maybe I could have done it in Word, but didn't seem like it was a good idea.

I know I did it on [00:45:00] Overleaf and you know, I love that you can really make it look the way you want it to look and put things there. And it's, I, I'm really happy with the way the Preprint looks, but might have been quicker to just write everything a word and then copy and paste it over.

Dan Quintana: Yeah.

Benjamin James Kuper-Smith: Um, because I think Word is still, despite all the criticisms, I think the best way to write something.

Yeah. Like there's all these fancy tools you can use, but I think, I dunno, at least for the, for the way I write something, having a folder on your desktop and a Word document, like a few do Word documents is usually for me, by far, the most efficient way of actually writing something. But I haven't written something with like 15 collaborators yet.

And just having like. I've had some, I've had some stuff where we were like, two people were print in parallel working on the same document, and that just was the most annoying thing ever. Um,

Dan Quintana: well, of, of course there's also Google Docs as a, as a way of collaborating as well, which, um, which is, ah, I use sometimes, but yeah, it's, it's, it's, it's not bad.

I still just, the way that track changes work, I still prefer Word, but, um, you [00:46:00] know, o other documents that, um, with other people I've, I use Google Docs as well, so it's, it's, it's, it's a bit of a mix, but look, every six months it's always different. Like, sometimes I'm like, yeah, overly fall the way, other times I get frustrated with it and then I'm back to Word.

But now it really just comes down to who am I working with on this particular paper. If I know I'm gonna send it to collaborators, um, who aren't comfortable with LA Tech, then I'll just do it in Word. If it's just me in my lab, then I might do it. I, I might do it in Overleaf. It, it really just depends.

Benjamin James Kuper-Smith: Uh, how does the, uh, I know Overleaf has this.

Um, I mean, so I'm just using like the free version and I used it basically to write it and then I export it as PDF and sent it to my supervisor. But they do have like a collaborate function in there, right?

Dan Quintana: Yeah. So you can,

Benjamin James Kuper-Smith: is that what you use? And if so, how well does it work?

Dan Quintana: It, it works. I, I, I, I still think the track changing just works better in Word 'cause.

Um, if you have a lot of comments, it can get quite jumbled very quickly. Um, but of course you can do the same sort of thing. You can, you can suggest edits, you can track change, you can, you can version control those edits and you can also write [00:47:00] comments. And then you can also have, uh, it's a nice feature.

I don't even know if a word has this, but there's like a general chat feature, so as well as talking about the paper, like directly, uh, you know about sections and there's also a chat feature as well. So you have like, you have sort of discussions about the paper within the paper too. Um, so it's. Yeah, it, it, it works well.

But the only downside is once you start having like a lot of collaborators with a lot of comments, it can be a lot harder. Word does a pretty decent job when you've got like, comments everywhere. Um, but Overleaf yeah, can sometimes yeah. Get it a little bit tricky to use. Um, it just depends on the project.

Benjamin James Kuper-Smith: Okay. I mean, I guess it some sense for me it's pretty easy because most of the papers I do is. I do the work and then we might have collaborators to help with one or another thing. Mm. Basically I write the thing, which I guess makes it a lot easier to just Yeah. Write the thing because I can just do however I want it.

And, um, yeah, just despite my, uh, strong distaste of most Microsoft things, well, it is [00:48:00] actually pretty useful. It, it's pretty good. It is still the best way.

Dan Quintana: W but word, word online I think is frustrating. Um, a lot of, I've

Benjamin James Kuper-Smith: never used that. Yeah.

Dan Quintana: It's, yeah. Um. It tries to be Google Docs in terms of its collaboration, but like it's, it just, it's clunky.

I don't know. I, I, I'm, I'm not a fan of, of maybe it, like, maybe it's changed recently, but, um, it feels very beta, beta ish. H how it works. Like curses will just jump and move everywhere. Um, but normal word, I don't know. It's just, yeah, it's, it's, it's easy to bash on Microsoft, but um, it, it, it, and

Benjamin James Kuper-Smith: I will

anyway, uh, yeah, we got through this via our, okay. Yeah. Somehow it's funny, like, I just assumed that, I dunno why it's, it's really funny in some, somewhere again. Uh, you said that you actually, can you confirm this? Is it true that you basically used SPSS for your entire PhD or that you only started using r on your postdoc or programming?

Dan Quintana: Yeah.

Benjamin James Kuper-Smith: [00:49:00] Oh,

Dan Quintana: absolutely. Yeah.

Benjamin James Kuper-Smith: Yeah. Like, because just for context, I've been listening to your podcast since, I don't know, like episode. 60 or something like that. I'm not sure. Okay. Nice. And for me, you know, you and James are these two people who are, you know, really into open science and refugee disability and all this stuff, so I just assumed you like, I don't know, grew up coding or something like

Dan Quintana: that.

No. Like, not at all. Look, look, James still uses Excel for a lot of his analysis. Yeah.

Benjamin James Kuper-Smith: Oh really? I actually, I, I found out, I'm not joking about three months ago that Excel is basically more than just a spreadsheet. I thought it was just like a place where you could like put data. I didn't realize you could actually like, do stuff with it.

Dan Quintana: Oh yeah. It's, it's, it's like, it's really powerful for, in certain contexts. But yeah. Look, I used SPSS because like. I had heard of R during my PhD. I graduated in 2013. Um, and back then, like I, I did, I didn't know anyone using R of course, I, I knew a few people, few [00:50:00] people using, um, MATLAB and R was just this thing that, um, I'm like, why would you, why would you ever do code when you can point and click?

Point and click is so much easier. But literally, uh, I had a, one of my papers during my postdoc, I needed to do, um, I think mixed linear models and you couldn't do that in SBSS at the time. So I'm like, gee, I be, I better learn this r this, I better learn this R thing. And then, um, really Well,

Benjamin James Kuper-Smith: that's good.

You didn't go, like, I'll just have a different experimental question then.

Dan Quintana: No, I, I just, I, I knew this is what I had to do, but the, the thing which really got me into R was meta-analysis using the metaphor package. Within meta-analysis was my gateway drug.

Benjamin James Kuper-Smith: Okay. Just briefly, I've, so I've never run a meta-analysis.

I, I think. I always also assumed you did r and that kinda stuff very early on because you did metaanalysis in your PhD, right?

Dan Quintana: Um, I did, but I, I didn't, I didn't use, I use different software.

Benjamin James Kuper-Smith: Ah, okay. I thought you somehow you'd, well, yeah, I dunno. I just Yeah. Assumed you used now. Okay.

Dan Quintana: So I, I'd use point and click software and the, the thing with this [00:51:00] point and click software is like, it's, it's not incorrect.

Like the, the, the software gives you the, the right numbers, but the flexibility that you get with R and the fact that you can actually share your analyses, um, makes it a lot better. And the thing I love about meta analysis is that you are working on public data, so you have absolutely no excuse not to share your data.

So whenever I'm reviewing a better analysis and I'm like, I wanna understand what this is doing, I always ask the reviewers, share your data. And they can never say no. Because it's, they have no excuse. The only possible excuse you can have is if you are somehow working with doing a mega analysis, for instance, where you are doing sort of individual you are, you are, you're doing a meta analysis based on individual data points.

But the maj, the majority meta analysis is done on publicly available summary statistics. So there's no excuse. So if you are doing this based on point, proprietary point and click software, you might share the data, but I still have very little idea of how you did your analysis. 'cause people are very bad at sharing their, their methods.

So unless they did a run of the mill meta [00:52:00] analysis with all the common defaults and even those things can, can change from, from lab to lab. It's very difficult. But if you're using r as well as sharing the data, you can share your scripts and, and then it, it, it's much easier. But this, yeah, look it doing and learning meta analysis within r if like, it's worth it alone.

So if you are doing meta-analysis, it's absolutely worth learning it. So you can do that. 'cause there's, of course there's the metaphor package, which has like some of the best documentation out there. Um, but there's also other. Meta-analysis, um, packages that you can use there. And, um, yeah, it's, it's, it's, it's totally worth it.

But yeah, look, I, I only picked it up during my postdoc and also it was from a lot of frustration from licensing from SBSS. Like, you know, you, you, you own the thing because you part of the institution, but then there's licensing problems. The software takes like five minutes to load. And through that kind of frustration, I'm like, well, you know, I'm paying you to my institution's paying you for, for a license, but you don't want me to use your software.

And it's, it's, it's with that, um. I went to R but um, look, it's, there's, there's nothing wrong with [00:53:00]SBSS, it's just, it's much more difficult to share. Um, and, and look, everyone's like, oh, the syntax, no one shares this syntax. You, you, you almost never see it. Ah. Um, but with r just makes it a lot easier. Um, but yeah, this, this, this sort of a, a recent thing that I've been getting into.

And I think when it comes to open signs and re reproducibility it, it's very important to stress that you don't need to adopt all these things all at once. There are so many things you need to get your head around. Open data, open scripts, pre-prints, pre-registration, and people think, oh, this is a lot of stuff and I can't learn it all for this paper.

I don't know what to do. Just do one thing for your next paper. Go. I'm gonna, I'm gonna share my scripts for your next paper. You're gonna think about, okay. Um, this is how I'm gonna do my preregistration bit by bit. If you're picking up these skills, um, it, that's, that's totally fine. Rather than sort of thinking, I, I can't do all these things things at once because it does, it's a valuable skill.

And the great thing is seeing more and more job advertisements that are saying open science and reproducibility [00:54:00] is valued. And it's valued because it takes time to learn these things and it takes time to learn these things. So you don't, you shouldn't feel this pressure of, I need to do this every, all these things for the next paper.

It's, it's difficult. Just do one thing at a time.

Benjamin James Kuper-Smith: Yeah, I agree. I mean, I've basically had to do all of this in my PhD and I mean, there are other reasons why, I guess some of my projects aren't published yet, but, uh, some of them may be more related to me, but one reason is also that I spend a lot of time Yeah.

Reading up on all this stuff, uh, like why, to visualize your data in all these different ways and not just use bar plots and, you know, like basic stuff like that. And there's like a million things like that. And I agree that. You know, in the same way that it's very, it's a lot of fun to, to bash word or to bash IPSS or whatever.

Yeah. That, that's maybe fun to do, but maybe not always the best way to actually get people to adopt it.

Dan Quintana: Yeah. Look, I shame isn't a great motivator,

Benjamin James Kuper-Smith: you know,

Dan Quintana: and a lot of people are very discouraged. Um, they open up Twitter and they see people look, and I'm gonna [00:55:00] admit I have been that person in the past to who, who has sort of somewhat unfairly bashed SBSS.

And in retrospect, I realized that is, that is not the best way to go, but the best way to go is actually. Pointing people to, to resources and how to actually learn these things, you know? But, but of course, like it's just, it makes it very difficult to share your data that, that I'm not gonna apologize for.

Makes it very difficult to, to share your scripts if you don't have, like, one, one person actually did for a paper, I, I was reviewing share their, their syntax and I'm like, oh, damn, I to do now I have to, I have to load S-P-S-S-I haven't done this in such a long time. Um, 'cause they're doing everything right.

But at the same time, I, I, I said, I think I said my review, well, you know, I have SPSS, but, but a lot of readers don't. Um, so if you really wanna make this more reproducible, then then really strongly consider doing this sort of thing in R or in jas or in, in, in Jamo, so other people can learn it. But, um, yeah, look, it's just this snobbery with different software packages and the ways of doing things.

As long as you're not making any errors, then, then I think I, I think [00:56:00] it's fine. But some software packages you, you're gonna be more likely to make the errors. So yeah, it's important to make that distinction.

Benjamin James Kuper-Smith: Yeah. And I guess it's also more of a long-term versus thinking, right? I mean, I think. Sure. It took me, I feel like I only read it after about one and a half years of my PhD of really programming cheap relatively frequently that I actually started to feel comfortable with it.

And, uh, now I can, you know, do my analysis without thinking about it too much.

Dan Quintana: Yeah.

Benjamin James Kuper-Smith: Obviously I have to start stuff, but it's, it's now fairly easy. There's a startup cost for that, but then again, you know, now it's super easy to run an analysis and show people what they did and change small things. And I think if anyone who wants to like.

Stay in academia for at least a few years. I think it's probably a worth inve investment.

Dan Quintana: It's a skill you need to learn. And even if you, I think it's also important to recognize that for a lot of people, academia isn't the way to go forever. And people want stuff in, in industry and in industry. Um, Python especially is, is within data science is very well valued, um, are as well to, to a lesser extent.

So [00:57:00] picking up these skills is, is gonna make you more valuable. Um, outside, we, both within academia, but also outside. And, um, you know, there are still a lot of departments that are still teaching SPSS, like I said, you're still gonna get the same results, but more and more departments switching across to r and um, people sort of assume, oh, the students are gonna hate this, but it's a massive assumption.

I've seen survey after survey of students that are like, this is super cool and really, and really actually enjoying, um, um, enjoying learning how to do r.

Benjamin James Kuper-Smith: I have to admit though, I think I kind of agree with the, let's say intuition or assumption that students would hate it because I probably would've hated it.

Because I think, I mean, you know, this is in part why I think also had spent so much time during my PhD in particular relearning stats and all that kinda stuff is because during my undergrad I was like, well, is this significant? You know, is it below or above 0.5? Like I don't care like about it. Like just tell me what it is and then that's all I need to know.

And only like now once I actually go like, yeah, but what did run results really mean? And like all this [00:58:00]kind, you have to really think about it. Then you go like, yeah, I think you have to learn stats properly.

Dan Quintana: Yeah.

Benjamin James Kuper-Smith: Um, but I, yeah, I think I'm always also the bad example to. Extrapolate from, because I think I'm just not someone who really likes sitting in class and doing what I'm talking.

So maybe, I mean, yeah, we had this course, we taught on the weekend. I was really surprised that the students were engaged. Like they actually like did what I asked them to do. What's wrong? What's wrong with you? People like, yeah, but maybe that's just me anyway. Um, oh wait, I just found a link between synthetic data sets and sounds communication.

Dan Quintana: Let's do it.

Benjamin James Kuper-Smith: Um, okay. Uh, here's my very elegant transition. I hope I'm getting this right. You live. You broadcast yourself writing this, the primer, right? That's right. With that paper.

Dan Quintana: That's,

Benjamin James Kuper-Smith: that's

Dan Quintana: the one.

Benjamin James Kuper-Smith: Sweet. So, um, I have to admit, I haven't watched that yet. Although I, I will be, I wanna see you like kind of what, what you're doing there, what, what that's like.

Dan Quintana: You see sweating, and sweating and swearing as I, as I make all these [00:59:00] errors.

Benjamin James Kuper-Smith: Sounds great. Um, but kind of you've, I mean you're doing quite a lot of science communication, right? Yeah. And this is one example of it. Yeah. And you know, I mean for, I try to kind of, you know, before I kind of do these interviews, I look at like, what can I find about the guests?

And I interview a lot of people who've, who have basically no online presence, um, at least that I can find. And, uh, for you, I found. You want to read the list? I don't. I found, okay, then I'm gonna do it. I found Twitter three podcasts. You've obviously been guests on other podcasts. A YouTube account, a medium blog, a separate blog called DS Quintana blog.

Then your website, ds quintana.com, university website, LinkedIn, Google Scholar type ResearchGate, frontier Sleep, Instagram. And lastly, I'm subscribed to your newsletter for about the last year. I think I've received one email. What's going on with your newsletter?

Dan Quintana: I, I have been a little bit, a [01:00:00] little bit stuck on the newsletter,

Benjamin James Kuper-Smith: but it was a eny, it was a very long newsletter, it seemed like.

Yeah. I was like, also, because I think I hadn't received an email for like a few months as I'd completely forgotten. I subscribed and so I was like, huh, Dan Quin is emailing me. Why that? And I was like, oh, the newsletter. That's what newsletter, that's right. And then it was just a, a wall of like, here's all the stuff I published.

But anyway. So kind of what's the, maybe as a kind of bigger picture behind like. All of these things, right? Is there kind of like one, either not like a specific reason for why you do all of this, but Yeah. I mean this is a lot of work, right? I mean, I do one podcast and this is a lot of work. Um, and you do all this other stuff too.

So kind of why

Dan Quintana: it's not as much work as you would think, because a lot of the stuff that I do, I repurpose for different mediums. So for instance, what I might do is if I'm doing a presentation, I'll record that as a video. I'll record that as a YouTube [01:01:00] video, and then I'll post on my YouTube, and then I can strip the audio and post that as a podcast.

And then I can take the best bits and best ideas and turn that into a blog post. I can take photos during the presentation and post that on Twitter and on Instagram or on TikTok or whatever. And,

Benjamin James Kuper-Smith: oh, you have TikTok? I didn't know.

Dan Quintana: Yeah, yeah, yeah. That's, that's one thing that you missed. I thought. I thought that's what you were gonna finish with.

Benjamin James Kuper-Smith: I al I think I also missed Facebook. I dunno if you have that. I don't have Facebook, so I couldn't,

Dan Quintana: I do have a public facing Facebook account, um, that, uh, and look, the other, the other thing is people like reading information in different ways. I'm a podcast guy. I almost never use YouTube for, for learning stuff.

I would much rather learn about a thing via a blog post, but people like taking in information in different ways. That's why I've sort of spread myself across these different platforms. The time I spend, the place I spend the most is, is Twitter. 'cause that gives you the most bang for your buck. Um, Facebook I don't like because the way the algorithm is, you basically [01:02:00] have to pay money in order to get yourself in front of people regularly unless they're already subscribed to your page.

Getting organic reach, which is basically reaching people without paying for advertising, is extremely difficult. Whereas getting that reach on Twitter is much easier in that it takes, it's, it all it takes is one person to see your tweet and to retweet it and offer goes into the stratosphere. That stuff rarely happens on Facebook.

So that's why I like Twitter in that respect. But look, all this stuff comes down to the fact that the way that academia works now is there's gatekeeping. Everywhere there's gatekeeping in terms of you submitting your paper and, and getting it published in, um, in, in journals. When it comes to you talking about your research with traditional media outlets, whether it's tv, whether it's radio, whether it's the newspapers, there's a lot of gatekeeping going on, which can make it very difficult to get your work out there.

If you are already famous, it's very easy. You have a new paper, you can call your contact the [01:03:00]newspaper, and away you go. If your mentor is famous, for instance, that can also really help. And if you can get very lucky in the peer review lottery and get a paper in that fancy journal, then a lot of people will start paying attention.

But other than that, it's very difficult for an early career researcher to get themselves known otherwise. But with social media, you can get around that gatekeeping if you have ideas. That you wanna put out there, you tr traditionally you would need to use traditional media or you would need to publish papers.

But with social media, you can get around that. You can start blogging, you can start preprinting, you can start talking about your ideas. And there is none of the gatekeeping that you do see with the, with the traditional journals and the traditional media. And there's just, uh, fantastic opportunities as well to learn new stuff.

We, we spoke about before me relearning statistics in my postdoc. A lot of the stuff that I learned was from finding stuff on Twitter, was from asking questions on experts on Twitter. How do I do this thing even, right. [01:04:00] Even

Benjamin James Kuper-Smith: I found so much about open science on Twitter. I think for that is especially useful.

Yeah,

Dan Quintana: it's huge. Hugely, hugely

Benjamin James Kuper-Smith: useful. And I don't, I hate Twitter. I don't really use it much and I've still found so much Yeah. About open science on it. Yeah.

Dan Quintana: E even like following certain accounts or asking questions, you can find a lot of stuff. And also, or just for those connections that you get. Yeah, I have a young family.

It's very difficult for me to travel overseas for, for, for, for a long amount of time. Free, you know, when I was, um, without kids, it's very easy to sort of go to different conferences and meet people. And I also recognize that for a lot of people, they don't have the resources to be able to do that, to be able to travel.

But with social media, with Twitter in particular, then you can get those opportunities for actually meeting new people and networking with new people. I've, I've lost count of the amount of papers, collaborations that are coming about because of Twitter, because of talking with people there. Invitations for doing, for doing talks.

That would never would've happened otherwise because people have heard the podcast. Um, 'cause people have seen me on Twitter talking about stuff.

Benjamin James Kuper-Smith: Hello? Yeah.

Dan Quintana: Yeah, exactly. Um, and like, 'cause we, we always [01:05:00] see from. All these conferences we go to, it's like, oh, that old white guy. It's like, did, did a keynote.

The last three conferences. The reason people do that is that people are very risk averse when it comes to talking, when it comes to choosing their speakers. And they're like, well, I saw that guy speaking at the last one. Um, one on white for the next one. But if you have a podcast, if you have a voice within social media, then people can get familiar with you and then you can become the safe option and they can contact you going, Hey, we want you to speak at this conference.

And then things kind of snowball from there. So I like it because it is a great leveler in many respects, and it levels the playing field, especially for early career researchers. And there's a lot of pushback from some senior academics going, oh, I don't like all this discussion on Twitter or this critical discussion.

Yeah, of course you don't like it because previously Yeah, exactly. You, you couldn't get challenged. You, you write a letter to the editor, but the editor's, your mate, and they're not gonna publish the paper. But on Twitter, look, I'm not, I'm not gonna say there's a lot of problems. It's not this perfect paradise, and there's a lot of stuff that can go on there.

Um, but at the same time, [01:06:00] um, there is a lot of benefits when it comes to, you know, sharing your work, learning from other people and, and building those contacts. And personally, it's been great for, for, for, for, for my career. And it's, it's just one we we're speaking before about, you know, using r and as as a new tool in your toolkit.

I think social media should be something in every researcher's toolkit as a way to talk about their research as a way to meet new people. It should be something that everyone, um, everyone is taught because it's just a, it is such a fantastic resource, I think.

Benjamin James Kuper-Smith: Yeah. Again, I have a bunch of questions based on a few things you mentioned.

Uh, first is maybe a bit specific, but how do you ask a question on Twitter that gets lots of response? For example, I, uh, created a Twitter account when I had my first. A half ago, whatever. I think I have 20 followers. I think I'm an, I think I'm an influencer now. I think I have 20 followers on Twitter. Um, so if I post something with a question, well, I don't, because I'm assuming no one's gonna respond because there's only 20 people who can even [01:07:00] see my tweets unless like someone with lots of followers happens to retweet it.

But that's pretty unlikely. If you have a question about something. How do you ask that on Twitter if you don't already have a big audience?

Dan Quintana: Uh, there's two ways. You can either tag experts, of course, don't spam them. But if there's an expert in the area that you think can answer your question, uh, it's, it's incredible for me.

If there's an academic that I know is active on Twitter and I wanna, I'm gonna ask them the question, I'll always contact them on Twitter rather than email them. Email. Email is just associated with drudgery. Twitter usually is a bit more fun, and these academics are much more faster to respond, whether it's a public.

Message, whether it's a private direct message, I'll always use that as a way of, of contacting academics. But uh, so one way is tagging someone who might have that expertise. Other way is by using certain hashtags, this doesn't always work, but for instance, I forgot to

Benjamin James Kuper-Smith: ask, does that work? I don't know. I've never looked for hashtags or anything.

So

Dan Quintana: I find that for R stats, for R. If you use the [01:08:00] ARS Stats hashtag, there are some bots which tend to retweet that stuff and in increase your reach. For me, uh, I've just seen it as well with a lot of other people who have had success with actually using the ARS stats hashtag. For instance, some research communities have their own hashtags that can help.

There's other ones, for instance, academic, CH academic chatter. Is a good one. So people who have, so with their PhDs, if they tag or hashtag academic chatter, then other people are sort of looking at these questions. I'm not gonna pretend that your questions are always gonna be answered if you're doing a thing.

A lot, a lot of people, I wanted to, I was doing a, a Twitter workshop, um, in Germany, where was it? Sick, I think, anyway, and I'm like, anyone who has less than 20 followers, who was less than 50 followers, and one person with their hand up, I'm like, we're gonna ask a question. We're gonna use the right hashtags.

And by the end, by the end of this session, you're gonna get a lot of answers. And I, I did it. We did it. And he got no answers. Like he was, if it worked, it would've, it would've been incredible. [01:09:00] So these things obviously don't always work. So I think what, what would be more successful is to tag one or two experts in the area.

Sometimes happens to me, people have meta-analysis questions and sometimes they get tagged. Sometimes I'm like, I don't know. And I'll retweet the question other times. Um, look, look, if, if someone can answer your question within the space of a tweet, this, the, the likelihood of them answering you is very high,

Benjamin James Kuper-Smith: right?

Right. Yeah.

Dan Quintana: So if it's literally, um, one, one thing I like using Twitter for is if I'm working on different scientific figures, I'm like, okay, A or B, what do you think? And then people will just vote on that, or they'll go, oh, A is good. But, um, you know, you should consider using different shading, for instance.

So that there, there's different ways that you can do this, but hashtags and direct tagging people are two ways that you can get your answers. Your, your, your questions answered.

Benjamin James Kuper-Smith: Okay. Yeah. I mean, yeah, as I said, it's not about, there's, there's no guarantee for any of this working, but I guess this maybe sounds very grim, but I just don't think if I just retweet something, there's gonna be any response.

Because, you know, as I said, I have 20 [01:10:00] followers like this, you know, that if I just do that, basically ask a question, then there's, there's just gonna be a response. But then with this, there's at least a chance then.

Dan Quintana: Yeah, exactly. Yeah.

Benjamin James Kuper-Smith: Uh, one thing I'd like to add to what you said earlier that this is. This kind of getting yourself out there and sharing your ideas and that kinda stuff, um, that, that's particularly useful for early career researchers just to hire.

I think it's especially useful right now during the pandemic. I mean, I've, you know, you mentioned conferences so far, two thirds of my PhD have been doing a pandemic.

Dan Quintana: Mm.

Benjamin James Kuper-Smith: And basically as soon as I had something that I could have shown pandemic and, um, there are, I guess still like online stuff, uh, like online conferences or whatever.

But yeah, I think especially like during this kind of thing when, um, I mean this isn't specifically why I started the podcast. I mean, it's a coincidence that it was around the time of the pandemic. But, you know, I get to meet every other week on average, like someone new, like, you know, like you for example now, who I wouldn't, I mean, maybe we would've met [01:11:00] otherwise, um, because of, because of Christoph.

But you know, many of my guests I would never have met otherwise. Yeah, I mean, I guess podcasts is maybe asking quite a lot because that is a fair amount of work, having some sort of podcast. But yeah, Twitter definitely seems like something that is doable here. Okay, so here's one question about Twitter just specifically, which is Twitter's, I find it annoying.

I'm not sure why. What's kind of the, what's the mindset I could have to her to start engaging more with it? I don't know. It feels to me still like an obligation, I guess. It is an opportunity, but it still feels like, you know, like it would be good to tweet more so I can have more following. So in case I have a, you know, when I have a paper, more people see it, but it doesn't really feel like something right now that I really enjoy doing.

Dan Quintana: It, it's really what you make it. So, and, and you don't have to follow scientists as well. It's like, like if there's other people, um, with within different spheres, you can follow them to make it more interesting. If there's something that you just can't stand hearing about, if you can't stand hearing about [01:12:00]cryptocurrency, you can mute certain words and tweets with those certain words will never pop up in your feed.

So there are ways of tailoring the experience to make it fun. And look, for me, the moment doesn't become fun. I'll stop using it. Like it shouldn't feel like drudgery. Like I, I don't have it, so I'm like, I need to tweet today. It's just one of those things that I'm like, okay, I'm just gonna. I'm just gonna share what I'm doing.

I, I never think about, um, I have some sort of tweeting strategy. Not at all. I'm just sharing what I'm doing and if, if someone is tweeting stuff I'm interested in, I just don't, I'll just stop following 'em, like, or mute them, you know? There, there's, there's no, there's no hard feelings there. And I'm sure people unfollow me all the time and that's fine.

Like, you know, it's just, that's just, that's just the way it is. But it should, it should be fun. And, um, one way to do that is just following the right accounts, I think as well. And as soon as it stops coming fun, I think about my use. I go through some periods where I delete it from my phone if I need to really, plus you

Benjamin James Kuper-Smith: have it on your phone.

Okay.

Dan Quintana: Sorry,

Benjamin James Kuper-Smith: I was curious, uh, [01:13:00] so you have it on your phone. Yeah. I was curious whether Accu, whether it's a use, whether it's a good idea to have TWI on

Dan Quintana: your

Benjamin James Kuper-Smith: phone. Well,

Dan Quintana: I mean, it's a bit, it's a bit more just, it can be a bit more distracting. Um, but for me, I'll go through some periods where I delete it from my phone and then I can only access, access it on a, on a desktop, which means I'm, I'm using it less and um, that's what I

Benjamin James Kuper-Smith: have.

Yeah.

Dan Quintana: Yeah. I mean, it's. It's, it's completely up to you as to as to how you use it. I, I wrote a book called Twitter for Scientists. It's a free book. Um, so I'm not sort of making any money from plugging it. And it goes through step by step how to use it. And one thing that I added to this book is a Twitter bootcamp.

It's a 30 day thing where I give prompts, suggestions of what people can tweak. 'cause sometimes people are just like, I just dunno what to talk about. Um, but within these prompts, um, you, um, you can actually go through and every day there's two different options and you can, and you can try. It's just, it's a way of sort of getting some, getting some momentum there.

Benjamin James Kuper-Smith: Ma, make a guess until what point I followed your book.

Dan Quintana: Did you, did you stop after the first chapter?

Benjamin James Kuper-Smith: No, no. I, it sort of got practical.

Dan Quintana: Ah, there you go.

Benjamin James Kuper-Smith: Until I actually had to do [01:14:00] something.

Dan Quintana: Chap chapter, chapter two, then, yeah.

Benjamin James Kuper-Smith: Really?

Dan Quintana: No, I, I think you,

Benjamin James Kuper-Smith: no, I read everything. Basically. I, I basically read the entire thing, but then when, um, it came to your, your bootcamp, I was like, ah.

Oh, okay. That seems like work. You,

Dan Quintana: you almost finished it almost.

Benjamin James Kuper-Smith: Yeah, exactly.

Dan Quintana: Yeah. So look, it, it should, it should be, it should be fun. Uh, other thing as well is like, people are,

Benjamin James Kuper-Smith: so, can I just add one thing to, to your book? I think the, uh, I mean, whether you call it a book or not, doesn't matter, but I think, uh, it's, it's quite short, right?

It's not like a book that you, you spend like five hours reading. It's like half an hour or something. Like it's, yeah, it's, you could have called it like blog posts, basically, but

Dan Quintana: it's all, it, it, it started being a blog post and then I realized there were so many sections that I converted into an ebook.

Yeah,

Benjamin James Kuper-Smith: yeah, yeah. Just so people know, like, this is not like something you, you know, that's gonna be a lot of work to read.

Dan Quintana: No, no. Like, yeah. Half an hour if that.

Benjamin James Kuper-Smith: Yeah. Uh, sorry, I interrupted you.

Dan Quintana: Yeah. So what I was gonna say is, the good thing is there are. Different types of social media that you can use based on what you're interested in and what you like doing.

Some people [01:15:00] hate video, some people love doing video. If that's the case, then yeah, Instagram, TikTok, YouTube. Um, some people prefer such as yourself, podcasts. I, I love podcasts. I think, I think they're fantastic and, um, but other people just can't stand the sound of their own voice, so then maybe blogging is a, is a bit better for them.

So there are different mediums and to that, that, that you can use that's sort of better suited to to, to what you prefer.

Benjamin James Kuper-Smith: Yeah, exactly. I mean, I think that's, yeah, that is one of the, yeah. Really cool things. Like whatever you prefer doing, there is a medium that probably only uses that specific way of communicating.

Yeah. Um, and yeah, I guess like even if you say like, I like writing, but I don't really have the time to write long posts more than use Twitter or Twitter's.

Dan Quintana: Yeah. Yeah.

Benjamin James Kuper-Smith: Yeah. Yeah. Maybe, um, to kind of maybe get a bit more toward podcasts. Um, I mean, you, you mentioned, uh, you, you don't like hearing the sound of your own voice, uh, but as you, I mean, as you said in your blog post on starting a [01:16:00] podcast or whatever, is there anyone who really likes listening to the voice in the beginning, at least?

I mean, in

Dan Quintana: the beginning, like, you, you, you, you learned at, at least for me, like I've learned to tolerate it. I'm like, okay, that's just, that's just how I sound. That's just how

Benjamin James Kuper-Smith: it's, yeah. Yeah, exactly.

Dan Quintana: But at first, at first, it's always very jarring. Um, but I think, yeah, it's an experience that everyone has.

Benjamin James Kuper-Smith: Did you, how long did it take for you? Because I was so, I have this real, I mean, I, I found it so difficult in the beginning, uh, in part this kind of, yeah, hearing your own voice and going, what? That's, that's what I sound like. And secondly, also, I mean, what I just found difficult was this idea like, this is final now.

Dan Quintana: Hmm.

Benjamin James Kuper-Smith: Like this, this episode is out there.

Dan Quintana: Yeah.

Benjamin James Kuper-Smith: And this is the way I sound. But I was really surprised to find that it took me maybe, I mean, I dunno exactly when, but at least like after episode 10 or something like that. So mine are like, you know, on average maybe like 70, 80 minutes long. So it's, you know, I listen to my voice a lot when after edited, but around that time I realized like, wait a minute, I'm not cringing anymore when I start speaking.[01:17:00]

I'm not sure I've got much better, but I'm not cringing at it anymore. How, how long did you take to kind of get used to it? Because since then I've basically been like, whatever, I don't care.

Dan Quintana: Yeah. About the same, about five or 10 episodes after that. I'm kind of like, yeah, this is just, this is how it is.

This is what everyone experiences. And, um, I'll just, yeah. Now I'm just, I'm just misused to it. I've done a lot of editing. I've edited myself. I can't even imagine how many hours. See, you just, you just, you just, you just get used to it.

Benjamin James Kuper-Smith: Yeah. I mean, that is a really cool thing. I think this, um, yeah, I mean, I guess, you know, not liking, listen to your own voice.

One reason. Of course, there's many other reasons why podcasting might not be the right thing. I think, you know, as you know, as I think most people are aware, it's probably easier if you're a native speaker. Oh, yeah. Yeah. It makes

Dan Quintana: a huge

Benjamin James Kuper-Smith: difference. Just as I said that, I think I stumbled over the word native speaker, but yeah.

Although for me, it's, it's bad because I, I, I'm only half a native speaker basically because I spent most of my life in Germany.

Dan Quintana: Mm-hmm.

Benjamin James Kuper-Smith: And depending on who I'm [01:18:00] talking to and who I spoke to before I started recording, the German can start creeping. Okay. Although I think my English is strong enough that people just think I'm South African then.

Dan Quintana: Okay.

Benjamin James Kuper-Smith: Because you have this like weird Germanic, British vibe, which then apparently sounds South African, but. Yeah, that's another thing that could be difficult,

Dan Quintana: but I quite like, I mean, a lot of people you, you mentioned before this idea of it's it's final, it's, it's, it's out there. But one of the reasons that I really like podcasts is it gives you the opportunity to get into some nuance.

It's very difficult for people to take you outta context. Uh, it's very easy for people to take you outta context on Twitter, on for a blog post. It's a screenshot. Look at what this person said. We've done 145 episodes and only once out of all that time, after almost a million downloads, has someone gone into the trouble of capturing the audio from one thing that, from one thing that we said we like, these guys are nasty.

What was

Benjamin James Kuper-Smith: it?

Dan Quintana: [01:19:00] Ah, we, we were talking about how somebody was disagreed with that there was, that there was an error in the paper. But it was just, it was maths. And I'm like, you, you can't argue with maths. Like two, two plus circles. Four. And we, we went and we were like, oh, this, this, the, the, it, it's maths. And we started laughing and then they, they took that as way of us, you know, oh, look, look, look at these guys making, making fun of papers, type thing.

But that was within, that was kind of within a, a, a wider conversation of, of people who were just, you know, disagreeing that their papers were wrong, but they were wrong. And yeah, one person's done that in that entire thing. Whereas with Twitter, it's very easy for people to take you outta context. When you're writing, it's very easy to do that.

But with podcasts you get that nuance and people, yeah.

Benjamin James Kuper-Smith: It's work to take someone out of context and a podcast. Yeah,

Dan Quintana: it takes a lot of work. You really

Benjamin James Kuper-Smith: have to, like, you have

Dan Quintana: to, this person worked, this person worked really hard to, to, to do that. So I like that. You can, you can really. Talk about certain ideas and, um, I think it's, it's much easier in writing a blog post because, because with a blog post, um, I [01:20:00] think people have, people have a lot more grace when you are talking.

'cause conversation. People have, people know what conversation is like and people know that conversations aren't perfect and people meander, they get to a point, they change ideas. You, you don't do that when you're writing

Benjamin James Kuper-Smith: or you could start like that, but you should probably edit. Yeah,

Dan Quintana: exactly. So there's the expectation to edit and have this polished, coherent piece with podcasting.

In a lot of sense, you are thinking out loud and people are totally okay with you thinking out loud and you even changing your mind admin sentence. It's, it's, it's totally fine. And in, in that respect, it takes, of course, editing takes a little, a little bit of time, but, uh, I think it takes all things equal.

I think it's much quicker and a much better use of time it's podcast than just a blog. I do blog from time to time, especially when it's a more technical thing, but when it talks about me sharing when, when I'm thinking about sharing my ideas much better on a podcast, much easier.

Benjamin James Kuper-Smith: Yeah, it's true. Even if I say like, editing takes a lot of time, it's like, well, editing writing takes a lot of time [01:21:00] too.

Oh

Dan Quintana: yeah.

Benjamin James Kuper-Smith: So, yeah, actually compared to that, it's not that much. Yeah, maybe as a, um, let's say, yeah, maybe let's say there's some listeners who are interested in starting their own podcast. Before that, before we get to that, I have one question. So there's a, there's a phrase that is, everyone has their podcast these days.

Do you know anyone who has a podcast? Because I don't. The only key point over podcast I know through my podcast,

Dan Quintana: very, very few people, there's one person in our department and she does a psychiatry podcast. Um, which is great. I think they have like an episode a month and they just talk about issues within psychiatry.

It's, um, it's Norwegian podcast, but, and yeah, and there's people sort of loosely connected and of course, but most of the people that I know with podcasts, I know via, via, via Twitter for instance. Um, there are other podcasts that sort of do similar stuff than what we're doing with Everything Hurts when it comes to methodology and, and reproducibility.

Um, as well with within sort of my, my workplace is just maybe, yeah, just like one person. I keep [01:22:00]telling people this is a, this is a great opportunity to, to do all this kinda stuff. Um, but you know, it's not everyone's cup of tea. I wish more people did it though. Like honestly. Yeah, exactly. Whenever I hear new podcast, new science podcasts, I'm like, yes.

Let, let's, let's grow, let's grow this community. I'm a bit biased 'cause I love podcasts myself. Like when I'm commuting, when I'm taking the baby for a walk, um, one hit, one headphone in and that's how I can keep on top of things. And, um, you hear some really interesting stuff.

Benjamin James Kuper-Smith: The baby's crying. Two headphones then.

Dan Quintana: Yeah, two headphones.

Benjamin James Kuper-Smith: But, uh, so actually just outta curiosity, who, who do you like listening to or who else, what are some kind of podcast that you really enjoy?

Dan Quintana: Uh, I enjoy, uh, quantity

Benjamin James Kuper-Smith: Other than your own.

Dan Quintana: I listen to it. I listen to it far too much in the edits. Um, I'm really, I'm really enjoying, um, uh, Ude.

Benjamin James Kuper-Smith: What's that about?

Dan Quintana: Um, so that's, uh, quantitative statistics. Um, the, the hosts hosts are fantastic and that's got a really good, um, production value. Like they, they spend a lot of time, it sounds corny, but like sound effects and like if they, if, if they like [01:23:00] reference a movie, they'll actually take the clip, the audio, like a five second audio clip from the movie.

Really, really cool stuff. Very, very highly produced, highly polished and very, very knowledgeable guys. I think the first podcast that I really got into for science podcast was, was Very Bad Wizards, um, which is sort of psychology. I'm

Benjamin James Kuper-Smith: sorry, what about, what was what?

Dan Quintana: Very, very bad wizards.

Benjamin James Kuper-Smith: Oh, very bad wizards.

Okay.

Dan Quintana: Yes. Um, it's a mixture of sort of psychology and philosophy. And these are two guys, um, that are just two mates who were talking. And I've been listening to them for a very long time and I realized that I do the same thing with James who was well before we started the podcast. 'cause he, he went to Boston.

I went to Oslo and we just kept talking, whether it was over Slack. Over email. Um, or just, or just a Skype talk. And I'm like, we're, we're already doing what they're doing on very bad wizards. Let's just, let's just record this. And that's how, that's how it really started. Um. I like other sort of more science ones like ologies.

It's just a cool sort of science [01:24:00] podcast. Um, what are the ones I'm listening to? Uh, I listen to a, I'm a bit of a Mac nerd. I listen to a few Apple podcasts.

Benjamin James Kuper-Smith: Like about Apple products or,

Dan Quintana: yeah, yeah, yeah.

Benjamin James Kuper-Smith: Okay.

Dan Quintana: So software and hardware.

Benjamin James Kuper-Smith: Yeah, that is nerdy.

That's,

Dan Quintana: that's, that's extremely nerdy. Um, uh, so yeah, that's usually the, uh, I, I like the, um, Ezra Klein podcast as well.

The old,

Benjamin James Kuper-Smith: well, I've heard about that, but what exactly is that? I've heard the name, but I dunno what it is.

Dan Quintana: Politics, sort of current affairs type stuff. Um, so yeah. But that, the, those are the main ones, but

Benjamin James Kuper-Smith: actually, yeah, I just read out. So I, I've had. This is now the second episode that I'm talking to someone who has a podcast also about podcasting.

The other is with Cody Commerce, who has the Cognitive Revolutions podcast or revolution. I guess it's one. I think he actually also mentioned Ezra Kle. Yeah. So I think now a hundred percent of the people Oh, great. Say they like that one. Maybe I should actually check it out though.

Dan Quintana: Check, check it out.

Just interesting people. Um, he's, the way that he ask questions is, is, is very [01:25:00] interesting also, like history podcasts as well. Um, there's, there's one, each episode is like three hours long.

Benjamin James Kuper-Smith: So the hardcore history,

Dan Quintana: hardcore history, that's the one, yes.

Benjamin James Kuper-Smith: Yeah. I dunno, I'm certain confused because I've looked for it.

And there on like, on, on Apple podcast, there's only a few episodes or something available,

Dan Quintana: I think.

Benjamin James Kuper-Smith: Is it?

Dan Quintana: Maybe it's a, it's a thing where, where you have to subscribe to get the older ones possibly.

Benjamin James Kuper-Smith: Mm-hmm. Okay. But that was even before Apple had,

Dan Quintana: oh, I'm not, I'm not sure

Benjamin James Kuper-Smith: this the, like payment method that's like half a year old or something, right?

Dan Quintana: Yeah.

Benjamin James Kuper-Smith: Um, yeah. I listened to one of them, but yeah, it's just so long.

Dan Quintana: I, I love it. Like when I'm on ho lying on the beach on holiday, I'm like, full of Rome, you know? It's, it's, and the way it's dramatized anyway, if you're in Indeed, I, I,

Benjamin James Kuper-Smith: I'm

Dan Quintana: sort of loosely interested in history, so that, that, that's a very good one.

Benjamin James Kuper-Smith: Okay. Okay. Yeah, that's cool. Um, actually, I, we have a, so one question I have is about, you can be as specific as you want to, but if you don't wanna share the specifics, you know, [01:26:00] you don't have to about kind of, for example, like the size of everything her, and how it kind of grew over time and that kind of stuff.

I mean, like, you know, you mentioned now you said almost a million downloads and I checked, there's this, I dunno how accurate this is, but there's this website called listen notes. It's just a website with lots of podcasts and it's, I think it's basically like a directory podcast where you can listen to podcasts, something like that.

They also have rankings, or not ranking exactly, but let's say kind of the top X percent. The popular podcasts are, and apparently everything hurts on the top. 1.5%

Dan Quintana: kidding.

Benjamin James Kuper-Smith: Podcasts on the website. That's a

Dan Quintana: surprise. That's amazing.

Benjamin James Kuper-Smith: And so the thing is, I, I know that Buzzsprout, the company I use for, um, uh, my hosting, they have stats about all the podcasts that they publish and what the downloads they get.

And I think the top 1% get like 3000 downloads in the first week or something like that. So just based on that, it seems like you're getting quite a lot of downloads per episode right. By now

Dan Quintana: We, we get about. I'd say about [01:27:00] sort of five, 10,000 a week. But 10,000 is

Benjamin James Kuper-Smith: in total, like across all episodes.

Dan Quintana: Across all episodes,

Benjamin James Kuper-Smith: yeah.

Dan Quintana: Um, a given episode within the first week would be sort of around four to 5,000. Okay.

Benjamin James Kuper-Smith: Pretty good estimate then. Yeah.

Dan Quintana: This like, we, we, we started off very, very slow. We started off as a psychophysiology podcast 'cause that's what we both did. And then, um, we realized there's about like 10 people in the world interested in what we do.

So then as soon as we started doing more episodes around research methods and, and life as another career researcher, that's, that's when a lot of interest sort of kicked up. But, um, yeah, I mean look, I, I think things really started kicking off around sort of episode 50 and then it sort of rises and rises from there.

And, uh, do you know

Benjamin James Kuper-Smith: why? Just because that's also rough. I mean, I was a bit late on that from listening to yours, but,

Dan Quintana: uh, I don't know. I think, you know, it takes time for, for, for momentum to start, uh, or for momentum to gather. Also, I think we began to [01:28:00] hit our stride a little bit more, became a little bit more confident with what we were doing.

The sort of format that we do, kind of crystallized a bit more by then. And we also started getting guests around episode 50. I think it, it was around then. Uh, and of course, you know, guest episodes tend to have, um, increased downloads, typically, not always, but typically they, they tend to have increased downloads.

And, um, yeah. So then I think even since then, like it sort of really increased around episode 50 and then it's just slowly, slowly increases. So we, we don't, we, we haven't seen that kind of explosion that we saw around then. But, um, you know, we, we still con continue with that rate of about sort of 10,000.

Also downloads a week. And, um, new episodes sort of get sort of anywhere from sort of like five to five to 8,000 downloads from, um, yeah. So it's, it's, it's, it's cool. It's really encouraging to see, we get some like really nice emails from people. Um, um Right. Thank

Benjamin James Kuper-Smith: you.

Dan Quintana: Yeah, it's, it's really cool. Like it's, I don't know, we, we never expected that we'd sort of get past 10 episodes.

We kind of said to ourselves, let's just [01:29:00] try for 10 and, and see where it goes. And, um, yeah, I think we're 4, 4, 4 years in episode a hundred and I'm, I'm just editing, I think 1 46 or 1 47 at the moment. And, um, it's, it's, it's been, it's been, it's been a great ride.

Benjamin James Kuper-Smith: I mean, do you, do you sometimes just, you know, the classic like, you know, sit down and go like, hang on, 5,000 people are gonna listen to us in the first week.

Dan Quintana: Yeah, it's still, I mean, look, look

Benjamin James Kuper-Smith: like that's, you know, just, you know, classic, like that's, uh. That's a big theater, right?

Dan Quintana: It's, it's, it's

Benjamin James Kuper-Smith: pretty, it's more like stadium size almost now. 5,000. Like,

Dan Quintana: well, I mean, look, look that, that, that's the amount of people who were downloading it to their device. We, we dunno.

Benjamin James Kuper-Smith: So it's like five people.

Dan Quintana: It could be, I mean, 'cause a lot of people just, it just, it's just an, you know, it just kind of happens and they never actually, um, listen to the thing. Um, that's also, it's a, that's a blessing and a curse of podcast is the metrics are very weak. We only know how many d devices are being downloaded to where they're located and the type of devices, which is very [01:30:00] good.

Um, there are some companies that are really trying to monetize podcasts, and by doing that, they're trying to force you to use a podcast app because the podcast app is getting information and is basically telling the creators, you, you lost attention at minute five, be edgier. And it's, it's just, it's changing podcasts, you know, for, for, for the worse.

The, the fact that the metrics are very. Fraud, I think is very good. 'cause it's meaning people are just making what they wanna make. Once you have too many metrics and people will sort of, not, like with YouTube, for instance, if you look at the metrics, you can see at what point of the video people, people started logging off and then people start tweaking their thing and becoming more, you know, doing crazier stuff in order to keep attention.

But with podcasts it's, it's staying, it's staying pure, so to speak, because those metrics are not.

Benjamin James Kuper-Smith: Sorry. Yeah,

Dan Quintana: go on. Yeah, yeah. The, the, the metrics aren't influencing people.

Benjamin James Kuper-Smith: Yeah. I think, I mean, one thing that I've heard is that, and that seems to be roughly true also for my episodes, is that podcasts do have really high retention rates, though over time.

Um, relative, especially I [01:31:00] think, you know, if you look at YouTube's graphs, apparently it's pretty much exponential, like fall off in the first 10 seconds or whatever. As much people realize, oh, I guess you have a YouTube channel. Yeah, you, you probably know this much better than me. Um, but for podcasts it's, I think, much more stable.

And I dunno, have you looked at the Spotify stuff? Because Spotify gives through Buzzsprout, I can, um, they do this fairly automatically where I can look at the stats. It's just in the first three episodes, it seemed to me to be pretty correct, where you saw like nuanced, like changes over time. But somehow for me, over the last few episodes, it's been like.

You know, it says like, I have X amount of listeners and all of them, you know, have this, you know, like, listen to the entire thing or listen, you know, whatever. Like 70%. Yeah. It's just, it doesn't look like real data basically anymore.

Dan Quintana: I haven't look closely at the stats. I think maybe 5% of our listeners listen via Spotify.

Five to 10% maybe.

Benjamin James Kuper-Smith: Oh, really? I have quite a lot actually. Okay.

Dan Quintana: I think, I mean, yeah. 'cause we, we kind of, we don't sort of promote it heavily. We kind of say, Hey, hey, it's an option. But, um, 'cause a lot of our [01:32:00] people began listening to us because they subscribed through their, through their apps that they already have.

Um, maybe more, maybe more recent listeners are, are, are going through Spotify. Um, but I haven't actually looked closely at the data there to see, to see when are we, when are we losing our listeners? Yeah.

Benjamin James Kuper-Smith: Yeah. I wonder because sometimes I think one thing that is a bit annoying about podcasting is that you've really lacked the kind of infrastructure that you have for videos through YouTube and.

Sometimes I think YouTube is amazing because you get stats. You know, like let's say you do the same thing every time and you think it's cool and everyone hates it. Maybe you should keep it in because you think it's cool, but, but maybe you should take it out because it sucks, you know? And um, I guess if you get no feedback, you never know.

But, and also the fact that, you know, with podcasting you don't really get even proper numbers about anything because you know, some people listen through that app, some through that, some through this, some through that, right. Sometimes I do wish, like, can't someone please make like a proper YouTube for podcasts?

But [01:33:00] yeah, I guess it does have downsides to.

Dan Quintana: Mm-hmm.

Benjamin James Kuper-Smith: One question I have about, yeah, I mean, so finances is something I don't really need to be thinking about much with my podcast in terms of sponsorship page or whatever. I just don't have the download numbers right now for that to make sense, I don't think.

But I'm just curious how you. Kind of think about that topic. Um, I think you mentioned in your blog post you started using Patreon from episode 75 onwards.

Dan Quintana: Mm-hmm.

Benjamin James Kuper-Smith: Or something like that. Yeah. How's kind of your experience been with Patreon and kind of monetizing the podcast? I mean, yours is maybe to say upfront, what's the word?

Uh, that you don't make nonprofit basically, right?

Dan Quintana: Yeah. So all, all, all the, all the money goes back into, to the podcast

Benjamin James Kuper-Smith: minus mine would be non-profit in the very technical sense, but, or well non-profit, I guess. If it costs me every month, that's also no profit. No

Dan Quintana: profit. Um,

Benjamin James Kuper-Smith: yeah. But yeah. So how, how kind do you think about starting to get some money from listeners through [01:34:00] this or through advertising and, and maybe also, um, if you are willing to slash can talk about this, some of your sponsors you've had, how did that happen?

Did you contact them or vice versa, or Yeah. How does that work?

Dan Quintana: I think we've had four different sponsors and all of them contacted us, I think, because people within their companies listen to the show and they're like, this, these are, this is the sort of audience that we want to get to. So they, they got in touch with us, but I think out 140 episodes, I think maybe 20 have been sponsored episodes, so to speak.

Benjamin James Kuper-Smith: Oh, really? That okay. So I thought it was okay. I thought it was more, but okay.

Dan Quintana: No. So, um, yeah, we haven't had that many, so maybe 20 or 30. And, and for us getting those sponsors is, is a bonus, which means that we're very picky about who we actually choose.

Benjamin James Kuper-Smith: Sorry, the, the sponsors are sites. Wait, then you've had prolific, right.

Dan Quintana: Prolific site, paper pile. Um. And I know I'm

Benjamin James Kuper-Smith: another very important sponsor. I know, I know that you [01:35:00] value dearly. I

Dan Quintana: know.

Benjamin James Kuper-Smith: Dan, I'm giving you so much. Come on. I can't say much more about how much you love that sponsor.

Dan Quintana: Our, our, our current sponsor is site, um, so

Benjamin James Kuper-Smith: Okay. Let's, yeah.

Dan Quintana: Site, site, paper pile, um, prolific and, um, another one, which, which, which, which we've had.

Um, but look, so, so we have that, and then we also have our, um, patrons, I think. We waited a very long time until we actually asked people. And I think what a lot of people forget is that some people really wanna support the show and there's no way for them to do that typically, other than from sort of talking about the, the show.

The show on social media tweeting

Benjamin James Kuper-Smith: about it at best. Yeah.

Dan Quintana: Yeah. And once you actually give that opportunity, you'll be surprised how people want to just go, Hey, I'll, I'll, I'll, I really enjoy what you're doing and I wanna give a little bit back. And that's been really good. And what we do is we release two episodes a month, and those are always free.

Those typically from sort of 45 minutes to a bit over an hour and as a bonus to our [01:36:00] patrons. Then we do a bonus episode, which is like 10 minutes, and that's what we sort of release. We also have like discounts on our merch. We, we, we do have merchandise, um, make very little money off that.

Benjamin James Kuper-Smith: I thought you said at one point that that's basically zero.

Dan Quintana: It's basically zero. Yeah. So it's literally like 10%, 5%. And for, for, for our, for our patrons, it's, it's essentially free, but we just have it there because some people will like, Hey, do, do you have t-shirts or mugs? And it costs us nothing.

Benjamin James Kuper-Smith: So do people actually use merchandise? I always find it really weird when people want like t-shirts.

I don't know. I guess I'm not someone who like, I guess you are wearing a NASA t-shirt right now, if I could tell that correctly. Yeah, yeah. So maybe I'm just the wrong person for this, but I never, I never get the whole like, I wanna own something in my house. People, people

Dan Quintana: do it. Yeah. People, people do it.

So I think our, our most popular things that we sell, um, um, are shirts, hoodies, and mugs. Those are the things that, um, yeah, and people just, people would want, people wanna support the show. I think that's great. But yeah, so all, all the money that we make goes back in. Um, it goes for, for equipment, be it [01:37:00]hardware and, and software.

Um, sort of, you know, it's just, you can do podcasts on the cheap or essentially free, but having some software makes it a bit quicker doing, doing it that way. So it's been nice, but it's also good that just to give listeners a, a, a way just to show their appreciation for, for what we're doing on the show.

And, um, yeah, Patreon's been great and it's just, it's another way for people to contact us. 'cause there's sort of like, there's like contact within, within, within the Patreon app. But yeah, look, I'm, I'm very glad that we waited until, until episode 70 and I, I think maybe. Two thirds of our patrons base that we have now.

We've been doing it for about two years now. Two thirds signed up within, within the first week. These were the, these were the hardcore fans. Oh, really? That had been listening. That's,

Benjamin James Kuper-Smith: that's cool though, right?

Dan Quintana: Yeah. Really cool. That must

Benjamin James Kuper-Smith: be great to

Dan Quintana: like that, that initial, like we, we, we, we had no idea whether people would actually, we're thinking maybe we'll get, maybe we'll get 10 people and maybe we'll just be able to cover the hosting costs and we'll just be able to cover [01:38:00] getting, getting new microphones and then that, and that's it.

But we we're pleasantly surprised and, um, yeah, like we, we, we still get new people that sort of come on board, but the majority of our patrons were from the very beginning because we had sort of, we built 70 episodes of, of, of Goodwill, so to speak. And as soon you said, Hey, we're doing this thing, you can get bonus and bonus episode and you can support the show, then it just happened.

So it's, it's really, it's really nice to see.

Benjamin James Kuper-Smith: But So what do you do with that money? I mean the, I mean, I get like, you know, I obviously know that podcasting costs money, but I think it says. Just check, was it $345 a month or something?

Dan Quintana: Yeah.

Benjamin James Kuper-Smith: So I mean, assuming you're not actually getting that right, that's probably before Patreon takes their cut.

Dan Quintana: Yeah. So Patreon takes their cut, um,

Benjamin James Kuper-Smith: which is what, 30% or something? It's quite high. Or

Dan Quintana: not that high. I think it's 15 or something. Um, but uh, look, this, this, I have to, I have to, um, count this as income, so I have to put aside a lot of that for tax as well.

Benjamin James Kuper-Smith: Uh, you can't do that like separately. You have to do it As for you personally, kind of, or

Dan Quintana: Yeah, so I mean, I mean, I, I, I could have set up a separate thing, but [01:39:00] it was just.

It's too much of a pain. So this is re regardless. It is.

Benjamin James Kuper-Smith: So we're funding like Norwegians and Roads or whatever?

Dan Quintana: No, seriously. I have to like, it is like, it has, it has to be, it has to be treated as, as as income. Um, so there's just, um, setting se setting up a nonprofit within Norway, it's just, it's not suited for a podcast, let's just say that.

So, yeah. So Patron X cut then obviously, I, I have to put that as, put stuff aside for, for paying tax. Um, but then, yeah, so the, these things, hosting the podcast, hosting the website, paying for stuff. Um, we, we use, um, melon as a service occasionally for, for speaking with guests that costs money, the hardware, the, the microphones, all, all the stuff associated with that.

Benjamin James Kuper-Smith: Okay. I mean, that, to be fair, just the fact that you have to tax that makes a big difference. So somehow I assumed you'd, I hadn't really thought about this too clearly, but I, I just assumed like, okay, you're getting like, I don't know, two 50 to 300 of that dollars plus whatever your sponsors make. Per episode, [01:40:00] uh, per, per month.

But I guess if you, if you're taxing that amount, then yeah, then suddenly it becomes a lot more clear what yeah. What you're doing with that one again.

Dan Quintana: Yeah. Yeah. It's, um, it, it's, it's, it's one of those things, like, you, you can't really, I can't really argue it's a hobby. It's, it's, it's, you know, money's coming in every, every month.

But, um, it, so it's, it's equipment and, and hosting costs. Um, you know, one thing we wanna do is, um, we, we, we're now at the point where we can start sending out microphones to guests as well.

Benjamin James Kuper-Smith: I was gonna ask, are you doing that or

Dan Quintana: No, we, we want, we wanna start doing that now. 'cause now, now we're at the point where we can sort of, you know, even after tax and all that kind of stuff, um, looking at the numbers, it's something that we can, that we can start doing.

Um, at least sort of something on the cheaper side, but it's much better than, than doing a laptop microphone.

Benjamin James Kuper-Smith: I guess the good thing is also that. You know, I have at basically two episode, uh, two interviews a month, um, on average, but I guess you don't do interviews every time. Right. So that also makes it much more manageable Yeah.

For you to actually do [01:41:00] that almost all the time. Yeah.

Dan Quintana: Yeah. Um, so I think that's something that can, that can really help. But, um, but, but also, you know, the, the, the software, so I use, um, getting a little bit technical into the weeds. I use Isotope Sure. To do, um, to do my editing. And it's just, it's, it's fantastic.

It's, it's heavily automated. Uh, it's not cheap, but it's, um, it just means I can spend less time doing editing, for instance.

Benjamin James Kuper-Smith: Yeah, yeah, yeah. I was actually gonna ask like one, uh, um, not necessarily like, feel free to mention like, specific things you use because that's, I think very useful. I mean, just to clarify, I use Audacity still, and you can, unless you're doing something, like if you wanna do like super cool edited stuff, then maybe something more complicated.

But for something like this. Audacity does pretty much everything you need, but it's just a little labor intensive. But I was curious, do you still edit your podcast yourself then? I was, I guess I assumed you had also more income through nature than you do, but, um, why? I'm just curious because I guess at some point you, I'm assuming you have, you could, you and James could put [01:42:00] some of the personally aside to do this, and then you don't have to, you have more time, basically, right?

Dan Quintana: Um, I don't know. I, I, I still like having that control. We don't do much editing at all, so we, we don't, we hardly cut anything out, but I still like doing that. And I'm the one that's writing the summaries as well and re-listening to it and editing it helps me write.

Benjamin James Kuper-Smith: You listen through the entire thing or?

Dan Quintana: Oh, yeah, absolutely. Yeah.

Benjamin James Kuper-Smith: Okay.

Dan Quintana: Um, so then, and then, you know, it means that I can listen through and I, we, we talk about a thing. I'm like, oh, I have to actually get the link for that too. So I like having that sort of. Reminder about what, what we spoke about. 'cause sometimes it might be like a week or two from between when we actually do the recording from when I actually do the editing as well.

So it's good to, um, it's good to listen to to, to these things. Um, so I mean, you know, even if we had a lot more money then I, I still think I'd do the editing myself just because Oh,

Benjamin James Kuper-Smith: really? Okay.

Dan Quintana: Yeah. Um, just, just so I can sort of re, re-listen to the episode. Um, but [01:43:00] also if, if there is a bit, a bit that we do cut out, even though we don't do it that much, at least I know.

And have that for control, like Yep. That, that, that's gonna be the bit.

Benjamin James Kuper-Smith: Okay. It's interesting just because, I mean, I guess I also do a weekly podcast and my episodes are longer, so that just, I, I spend so much time editing, like at some point I'm just getting fed up with it.

Dan Quintana: Yeah.

Benjamin James Kuper-Smith: But I've, I've really considered.

You know, because I have the same thing that I, um, you know, I guess ours today was one of the least technical episodes I've almost done, but often I'll, you know, I, I have some episodes where I have like 20 references in the list or something because we talked about, you know, mentioned so many papers and whatever.

And obviously pretty much I have to do that. And, but there's some episodes where I have like book discussions or something where someone else could do it and I'm, I'm already, I might do this from like, next year on basically say like, someone please help me. I'll

Dan Quintana: pay you. Yeah. I mean, and you can, you can, you can totally do that.

But I mean, I think, um, right now it's not too much of a time Im position because, because we don't, because I don't do too much editing. Um, I very rarely take out pauses, for [01:44:00] instance. Or if, if there's a bit of the episode, if it sounds a bit awkward, we keep it in 'cause it's real. That that's what conversations are like.

And we always wanna give that impression of this is what a conversation sounds like. So it doesn't take. Super long because a lot of this, 'cause you know, fortunately we do have patrons, so I can pay for software that can automate a lot of the stuff which comes to, to, to fixing a lot a a lot of the audio.

And I save a lot of time in that way. So I'm very fortunate that we do have the patrons. So I, I can actually automate a lot of that stuff.

Benjamin James Kuper-Smith: Yeah. I guess maybe, uh, as kind of last point about editing for anyone who's interested in doing the podcast their own, I, I think I will say that one of the time consuming aspects of my job in that sense is that.

I have a new guest every other week at least, which means I have no idea what audio's gonna happen that week. Um, you know, I guess if you and James are doing an episode between you, or if I'm doing a book club with a friend, I know what the audio's gonna be like. But I've had some episodes where I spent like eight hours editing the thing because there was [01:45:00] like, sounds that were so like irregular and annoying and no, I just couldn't figure out how to do Makes a big difference.

Yeah. I guess that, yeah, that probably makes a huge, because those are the ones that kill me, like a two hour interview. Oh yeah. It sound is awful. And I

Dan Quintana: didn't take a lot. Yeah, I mean, we, we, the same thing, like some guests, um, don't have the equipment and that's, that's totally reasonable. Yeah, exactly. I mean, it mean it just takes a little bit of extra time to do that.

Those ones take extra time, but because I'd say three quarters of the episodes are just James and I then, right? Right. The, the editors are a lot, uh, the editors are a lot quicker.

44. Dan Quintana: Synthetic datasets, science communication, and podcasting

Featured Posts

Read Full Transcript