NeuroDiving

Amelia: Before we begin, I just want to let you know that this episode might be pretty confusing if you haven't listened to episode 2, at the very least . So if you haven't already listened to the previous episode, I suggest you go back and do that now.

Introduction

Amelia: In the early 1980s, Simon Baron-Cohen, Alan Leslie, and Uta Frith conducted an experiment. They recruited 20 autistic kids, 14 kids with Down Syndrome, and 27 neurotypical kids, and they gave all these kids a false-belief test; specifically, the Sally-Anne test.

For the Sally-Anne test, the kids watched a little play involving two dolls, Sally and Anne. To start off, Sally and Anne are hanging out together in the same room, and Sally puts a marble in a basket. Then, Sally leaves the room. And while Sally is gone, Anne takes the marble, and puts it in a nearby box. And then Sally returns.

At this point, the researchers pause the play, and ask the kids questions, like “Where will Sally look for the marble?” and “Where is the marble really?” To pass this false belief test, a kid needs to know that the marble is really in the box, but that Sally will look for the marble in the basket. And the researchers thought that if a kid could pass this test, then that kid must have good theory of mind—they must be good at figuring out the mental states of other people.

Here were the results of the study. 23 out of the 27 neurotypical kids passed the test; 12 out of the 14 kids with Down syndrome passed the test; but only 4 out of the 20 autistic kids passed the test. When asked where Sally would look for the marble, the autistic kids usually pointed to where the marble really was–even when the marble was in the experimenter’s pocket. To the researchers, it seemed like most of the autistic kids couldn’t distinguish between their own knowledge and Sally the doll’s knowledge. It seemed like the autistic kids didn’t grasp that Sally the doll held a false belief about the location of the marble.

So in 1985, Simon Baron-Cohen and his colleagues published a paper titled, "Does the Autistic Child Have Theory of Mind?" Their answer to that question was "nope"---according to their findings, autistic children had deficits in theory of mind. This conclusion offered autism researchers something they had been seeking for a long time: a unified theory of what's going on with autistic people.

Joanna: For most of the twentieth century, autism researchers didn't have a unified theory of autism. According to some researchers, like Leo Kanner, it was some sort of a developmental disorder; according to other researchers, like Hans Asperger, it was more like a personality disorder. And for years, autism was classified as an early childhood manifestation of schizophrenia. But in the 1970s, autism was officially classified as a distinctive condition in the DSM-III---the third edition of the Diagnostic and Statistical Manual of Mental Disorders. But even though autism was codified as a distinctive mental disorder in the DSM-III, it was still poorly understood. Autism just seemed like a cluster of different traits, with nothing to explain why all those traits clustered together.

Amelia: That's part of the reason why Baron-Cohen and his colleagues made such a big splash when they introduced the idea that autism is a theory of mind deficit. All of a sudden, it seemed like many different autistic traits could be explained by one thing: problems with theory of mind. This was a whole new paradigm for understanding the nature of autism, and autism researchers were pumped. But as we'll learn today, this research has always faced serious methodological and ethical problems.

This is NeuroDiving, a philosophy podcast about neurodivergence.

I'm Amelia Hicks.

Joanna: And I’m Joanna Lawson.

Amelia: Today, we learn about the strange history of research connecting theory of mind and autism. We'll explore the scientific problems with this research, as well as the harms caused by that research. Given all these problems, why has this research continued to be so influential? What's going on?

Replication Problems and Moving Goalposts

Amelia: For over a year now, Joanna and I have been exploring the history of research on autism as a theory of mind deficit. And this whole time, we’ve felt bewildered, shocked even. This research seemed to keep going wrong. And then, even when it kept going wrong, it never seemed to change course.

Joanna: Right. And, in the previous episode, we mentioned a couple different problems with theory of mind. First, the concept of theory of mind is confusing and ambiguous. And, second, there are problems with using false belief tests to measure theory of mind. But now we get yet another problem: researchers haven’t been able to reliably replicate that early research carried out by Baron-Cohen and his colleagues.

Amelia: In other words, when other scientists ran the same experiments but with different autistic people, they didn’t always get the same results. As it turns out, lots of autistic people pass false belief tests, and lots of non-autistic people fail false belief tests. When those early results from false belief tests couldn’t be reliably replicated, autism researchers had to make a choice: “do we abandon the theory of mind deficit view of autism? Or do we come up with a different test for theory of mind, so that we can hang on to the theory of mind deficit idea?” And because the theory of mind deficit view of autism was so attractive to researchers, they didn’t want to get rid of it. So they started coming up with new tests for theory of mind.

Joanna: There’s a really great paper by Morton Ann Gernsbacher and M. Remi Yergeau in which they survey all these different failures of replication in detail, as well as the new tests that followed each failed replication. At the end of their paper, Gernsbacher and Yergeau write,

Rach: "The development of more and more theory-of-mind tests resembles a methodological arms race. The deployment of first-order False Belief tasks escalates to second-order False Belief tasks, which escalate to the so-called advanced theory-of-mind tasks, and then to the Strange Stories Film task, the comic Strip task, and the Beauty Contest task---all in pursuit of finding a task to support the claim that autistic people lack a theory of mind, when previous tasks fail to support the claim."

Amelia: When researchers found that many autistic kids passed the Sally-Anne test, they came up with a new, slightly more complicated test called the second-order false belief task. In this new experiment, it wasn’t enough for kids to know when a character, like Sally the doll, held a false belief. This new task required kids to answer questions about what one character believes about another character’s beliefs. But then it turned out that lots of autistic kids could pass that test, too! And so researchers came up with even more difficult theory of mind tests—and this just kept going, and going.

Joanna: In other words, it sometimes seems like researchers went looking for ways of measuring theory of mind that would support what they already believed about autistic people, namely, that autistic people lack theory of mind. And that's pretty messed up--in good scientific research, you shouldn’t choose measurements based on the conclusion you want to support, right?

Introducing Tobi, and the Reasons for These Failures of Replication

Amelia: What the heck was going on? And why did this research continue to be so influential in spite of all of its limitations? Clearly, we needed to talk to an expert in psychology. Fortunately, we met Tobi Abubakare.

Tobi: Yeah, so, hi I'm Tobi. My full name actually is Oluwatobi, but most people just call me Tobi. And I am currently a–wow, I guess a rising second-year student?–in the clinical psychology PhD program at IU Bloomington, and I'm doing research within autism, particularly looking at how social interactions differ between autistic and non-autistic individuals and how that impacts social and health equity for these populations, specifically marginalized autistic populations such as those who are Black, um, other BIPOC minorities as well as, um, those in low-income and low-resource communities as well. Outside of that, the cool thing I do, I guess, is I am a violinist--I've been playing violin for, like, almost twenty years. I don’t know if I’ll stop. But it’s been cool to be a musician for so long.

Amelia: In addition to twenty years of experience playing the violin, Tobi has lots of experience working in psychological autism research. And Tobi uses their experience as an autistic person to enhance the quality of autism research.

The first thing we wanted to ask Tobi was why those early theory of mind deficit results couldn't be replicated. Like, what led to those failures of replication?

Tobi: I think some of the reasons why maybe the replication differences occur, um, it's primarily because of a couple things. So one, a lot of the studies that came out from, um, Simon Baron-Cohen particularly and his colleagues, you know, use a lot of small sample sizes. So, you know, when you have like an n of like 20, it's hard to then to be like well this n of 20 can be a universal way of understanding this entire population. I mean, one reason why that occurs is because it's really hard to obviously get participants, and that's just a fact within at least psychological research. It's, it's a grind um, as someone who's also doing a study now. So it is something we have to figure out how to be better at as, you know, psychologists and figuring out “how do we make better inferences with such small numbers?” And I think that could, uh, one of the reasons why this replication difference, um, could be.

Amelia: So problem number 1: most autism research focusing on theory of mind uses teeny-tiny sample sizes.

Joanna: For example, that original experiment that launched the idea of autism as theory of mind deficit only involved 20 autistic kids. With sample sizes that small, an experiment’s results can’t really be generalized–they just don’t have that much statistical power.

Amelia: Like, if you wanted to test the very obvious hypothesis that people who like eggs are more likely to report eating egg salad, you would still need way bigger sample sizes than what you usually find in these studies on theory of mind and autism.

Tobi: But another reason why this sort of replication issue occurs is because several of the studies that were trying to redo these types of understandings of theory of mind in autistic individuals found that there was a language difference. So, one thing that I think most researchers will agree with in psychology about autism is that there’s language differences within autistic individuals and that can influence how people engage in theory of mind, but also how people participate in these studies. Like a lot of theory mind paradigms, particularly like the Sally-Anne test, rely on verbal understanding. So you're speaking to these individuals and asking them, you know, "What do you think about x?" and then they have to tell you themselves their answer, and that requires some type of, like, language ability, to be able to comprehend the speaker's question and then able to, you know, say your answer in a way that the speaker can understand. That requires some type of language ability, and so that differs between a lot of these studies done with theory of mind. There’s language differences between the groups that were participating in these studies. And I think that could be another reason why this replication issue occurs, because people are using different samples of the autistic population to make inferences about the larger population, given that you have to be more specific. So I think one issue in general with autism research is that, when speaking about autism, you have to be very explicit about characterizing your sample.

Amelia: So here we have replication problem number 2.

Autism is an extremely heterogeneous condition--that means that one person's autism often looks totally different from another person's autism. As a result, even if you have a decent sample size, you can't always generalize your results to other autistic people; you have to be careful how you characterize your sample.

Joanna: In particular, autism often impacts a person's verbal abilities; one autistic person might be minimally speaking, and another autistic person might be hyper-verbal. Early theory of mind tests were often administered to children with verbal disabilities. So when those kids performed poorly on those tests, it didn't necessarily mean they didn't have theory of mind; it probably just meant that they had trouble with verbal tests. Go figure! And the fact that some autistic kids perform poorly on verbal tests doesn't tell you much about how other autistic kids will perform on those same tests.

Amelia: So we have these attempts to study the relationship between theory of mind and autism, and those studies had serious methodological limitations. As a consequence, the results of those studies were difficult to replicate. And yet, this “theory of mind deficit” view of autism stuck around for a long time. After each failed experiment, researchers would come up with a slightly different measure of theory of mind, in the hopes that that new measure would vindicate the idea that autism is a theory of mind deficit. What on earth was going on?

Why Did This Research Stick Around?

Tobi: This was a really big finding back in the 80s and 90s. Like, it was, I think, really profound that we found this big difference. But I think one of the reasons why this stuck around is because I think this type of finding really cemented what people perceived already about autism. That autism already is a deficit in social interaction, that autism is a deficit in social communication. From the perspective of a neurotypical person like “why not go with this finding?” because it matches our perception of autistic people in general. I think that could be a reason as to why, like, throughout the history of this type of work that it has stuck around so long because it confirms what people already assume about autism and just disability in general, that people who are not typical, who are not "normal,” can't do what non-disabled people can do. So therefore “we have findings that sort of justify this or confirm this; why not stick with it?” That for me, and I think for a lot of autistic researchers, and just autistic individuals in general, find that really discomforting.

Amelia: Part of the reason the "theory of mind deficit" view of autism has stuck around for so long, in spite of its poor empirical support, is that it reinforces common assumptions about autistic people. Researchers started off by noticing differences in how some autistic kids engaged in pretend perspective-taking; but they inferred from those differences that all autistic people have a serious deficit. And even when later experiments seemed to show that autistic people engage in many types of perspective-taking, researchers still clung to the theory of mind deficit view of autism. This view of autism reinforces the idea that autistic people are fundamentally disconnected from other people, and from ourselves. So, this is a case in which researchers make value-laden assumptions about autistic people, and those assumptions are leading to messed up research.

Tobi: I don't think I know what the scientific reasoning is. I think for me, I find the reasoning to lie within more critical theory. That there's this perception of what those who've been deemed as abnormal--so these who have been a pathologized for their differences, so whether it's disability in general or autism--that when these populations end up doing things that go against the grain of what, uh, story or narrative that has been created by the able-bodied society or population, it's like it doesn't register. It doesn't make sense because it doesn't coincide with what their beliefs are. The same way, uh, you know going back to like research and race, how throughout history there's been research--not great research--about you know, assuming that like Black folk are academically, intellectually inferior to white people. And then when Black people do things that go against this really bad racist narrative, it's like “no, it doesn't make sense. It can't be possible.” So, it's just like, I feel like it's less of a scientific problem and more of just, like, ethically how certain people are valued because of their certain abilities, and if you don't have that, you know, we are going to create some type of negative narrative and you're not allowed to exit out of that narrative. You're not allowed to supersede that narrative, because that's what we placed you in. And so when you do, it doesn't compute. So they do all sorts of, like, mental gymnastics to make sure that their theory holds.

Amelia: Sometimes, when we really want to believe something, we jump through all sorts of mental hoops in order to explain away any counter-evidence.

Joanna: It’s sort of like when early renaissance Europeans wanted to believe that the Earth was the center of the universe, and in order to explain away all of the counterevidence, they proposed that the planets were attached to spinning invisible spheres, which were themselves attached to even more spinning invisible spheres. Have a new piece of counter-evidence to geocentrism? Just postulate another spinning sphere! Sometimes we tie ourselves in knots in order to hang on to our ideas and beliefs.

Amelia: And we find similar mental gymnastics in autism research. After those early theory of mind studies didn’t replicate, researchers hypothesized that the experiments failed because autistic research participants were using their cognitive and linguistic abilities to “hack out” answers to theory of mind tests, without actually using theory of mind. The thought was basically, “autistic people have a deficit in theory of mind, so, they must be passing theory of mind tests by using some other ability to compensate.” Because of that line of thinking, researchers kept trying to come up with subtler theory of mind tests that autistic people wouldn’t be able to “cheat” their way through.

Tobi: And you know the idea of like "you can cheat your way into theory of mind" makes no sense because theory of mind, there's no exact way of doing it. I don't think I've seen research yet that has proven that there is an exact way to do theory of mind. I think these are just people's beliefs that, “oh there should one way to do it, because this is what we as neurotypicals have assumed it to be, this is how we do it.” So I think for me, it's more of an ethical issue than, like, a scientific issue because I think scientists value more, you know, normality, value more of what they deem as normal. And so I think it just backs up their value, what they have values in, rather than, you know, what the numbers are telling them.

The Harms of Theory of Mind Research

Amelia: It would be one thing if this messed up research was just sitting in an ivory tower, unread by anyone. But these ideas–that autistic people can't understand other people’s perspectives, and aren't even aware of our own perspectives–have filtered down through academia, through medical clinics, and into our everyday interactions. This is how the theory of mind deficit view has managed to cause so much harm. I asked Tobi how they feel the theory of mind deficit view of autism has impacted them personally.

Tobi: So, I guess one story I have is about how this one person I guess presumed that I lacked empathy or I lacked the ability to understand that they were grieving. And so in this situation, um, this person went through a series of deaths in their family and um, this person was someone who was sort of like a mentor to me. And so one thing I knew about this person that they really loved, that they really liked stuffed animals, particularly I think it was a bear, that was like their favorite stuffed animal, and so I went to an amusement park one day, uh, for something unrelated to this, and I was like “well this person really likes stuffed animals, let me see if I can win them a stuffed animal.” And I got lucky and I won them a stuffed animal. And so the next day I gave them the animal. And they were really taken aback. They were surprised, because I guess they assumed that I wasn't capable of sharing that type of emotion or just understanding that they were going through a hard time and so they were really touched by that, especially in the fact that they ended up, they told my mom that like, "oh your kid did this amazing thing for me," even though it didn't feel like such a big deal to me given that, you know, I just did something nice. But for me, it felt like people presumed that I wouldn't be able to do that because they had assumed that I lacked empathy or didn't show empathy and just because sometimes my face does have flat affect, so people sometimes just presume a lot of negative emotions around me because of their perception. So I guess that's a story of how people would presume I don't have a good theory of mind or have a good perception of other people's feelings or thoughts, because of the way that they perceive autism or autistic people to have.

Amelia: When Tobi gave their grieving mentor a thoughtful gift, the mentor was way too surprised. Obviously, when you give someone a gift, it's nice for the gift recipient to feel surprised. But in this case, the mentor was shocked that Tobi had the capacity to give a thoughtful gift; and that level of shock revealed the assumption that autistic people, including Tobi, were not truly capable of being thoughtful at all. Tobi says that they're not very upset or angered by this experience; it's just yet another daily reminder of the assumptions people make about them, and about all autistic people.

Tobi: Yeah, this is something I have struggled with not only as autistic person but also as someone who navigates many different identities like being Black and disabled. Like throughout my entire, like, childhood I assumed something was wrong with me that, you know, I wasn't someone people want to interact with because I was a bad person. Um, and that badness was because, um, socially I couldn't engage with people but also because, uh, I walk differently and so that making people to have to somehow accommodate me is bad. Because that's what I was being fed by, um, is that, like, people wanted to try as much as possible to cure me of my disabilities. Um, because it was a bad thing. So I thought I was a bad person. And so like, these things do affect people's perceptions, like internally.

Amelia: Tobi feels fortunate that they were eventually exposed to more positive, or at least neutral, ways of thinking about autism and disability. But even so, years of having people make negative assumptions about them has taken a toll.

Tobi: Where I'm like, “wait, the problem might not be me. The problem might just be society and how society has created barriers to make it seem like autism is bad when actually autism could just be another way of interacting and being in the world.” Um, and so I sort of got out of that mindset in college. But before then I had a negative self-conceptualization of myself, and that obviously affected a lot of, you know, my mental health. I was pretty depressed, um, in my childhood, um, especially in high school. So I don't, I don't think it helped. I think it hurts. I don't think a lot of autism researchers think about that, like, the way you talk about autistic people, the way your research gets framed in that. These people exist. We're not just, like, cells in a petri dish, we exist and therefore everything you say and how you write impacts us very much. Um, and so I just, researchers need to just think about this. They just need to think about what they're doing and the implications of what they're saying.

Amelia: Researchers need to think about the implications of how they speak and write about autistic people. Because it's not just Tobi who experiences the negative effects of sloppy autism research. This research filters down to our culture, in a way that shapes many of our everyday interactions.

Tobi: I think there's an impact of, like, personal impact, where because people have now assumed that if you say someone's autistic, you're changing your behavior towards them because you now have this assumption that they cannot understand you, that they cannot interact with you in the same way that, you know, you seem to interact with other people who are not autistic. That we are closed off and lack empathy, and so therefore you change the way you interact with us and you change your behavior. And that makes it more isolating, because then you're excluding these individuals and not understanding that it might not be that there is a theory of my deficit, but rather it seems the problem isn't their theory of mind but rather it's a problem of different communication styles and different expectations.

Amelia: Some researchers might say that autistic people experience isolation because autistic people have deficits in theory of mind. But according to Tobi, the theory of mind deficit view of autism contributes to isolation. People who believe that autism is a theory of mind deficit might not bother trying to communicate or form relationships with autistic people. And that leaves autistic people even more isolated.

Tobi’s Recommendations for Researchers

Amelia: Tobi has some recommendations for how researchers can do better. I mean, there's the stuff that’s now pretty obvious, like "use bigger sample sizes" and "characterize your samples more carefully." But Tobi's recommendations for researchers go way deeper than that. It's time for researchers to do some soul-searching, and to think critically about the values that are driving their research.

Tobi: So I think for psychologists something to just consider is that when we are searching for differences in our research, maybe take a step back and just think about "Why am I searching for this difference, and what do I want this difference to say?" And if I want the difference to say a negative thing, ask yourself "Why? Why should it be this negative assumption?"

So, I wish researchers could think about, well, “what is a different perspective I can have on theory of mind” that isn't just being stuck in this assumption that, like, you have to find a deficit because you've assumed already that this population is beneath you. And maybe realizing that, is it possible that difference doesn't equate to deficit, but difference could just be a difference? Like, a lot of psychological researchers, we are trying to find differences, significant differences, and then making some assumption about that difference, and it's usually never a positive assumption. It's usually a negative assumption, especially when you're talking about, you know, mental conditions, and this idea that people are somehow just different. Like we assume that difference is a bad thing, but difference could just be a thing. It doesn't need to have some type of value on it, it could just be a thing. In the same way, like, violins and violas are different from each other. That's neither a good or bad thing. They're just different, they make different sounds and they're just different, and that's fine.

Amelia: I don't think that the problem here is that researchers allowed values to infect their science. Instead, I think the problem is that researchers' values infected their science without the researchers even realizing it. It's inevitable that our values will guide scientific practice. After all, our values influence what we decide to study in the first place! But we need to carefully reflect on our values, and be intentional about the roles we let our values play in our scientific research. But what might more reflective autism research look like?

Look-Ahead to Next Episode

Amelia: Next time, on NeuroDiving:

Travis: Uh, yeah, I think that some of the sort of empirical work that we get suggests that, at the very least, a theory of mind deficit theory of autism is bad science. But in this case, I would think that theory of mind must be degenerating, and even to the extent of being pseudoscientific.

Amelia: We will dig even deeper into the philosophy of autism science and, dare I say, pseudoscience.

Credits

Amelia: This episode was written, hosted, and produced by me and Joanna Lawson. Another big thank-you to Rach Cosker-Rowland for her voice work, as well as her editorial advice. And many thanks to Tobi Abubakare for helping us sort through all the problems with theory of mind deficit research. Tobi recently published an amazing essay in the journal Autism in Adulthood, titled “The Unexpected Autistic,” which you should definitely go read. Tobi is also a member of the Black Empowerment in Autism Network, which goes by the acronym BEAM. BEAM aims to provide opportunities for Black autism researchers and experts to connect, share their experiences and knowledge, and discuss ways to improve understanding and support for Black autistic individuals. In our show notes, you can find a link to a recent press release about BEAM’s first meeting, which took place this past summer. Also in our show notes, you can find links to the papers mentioned in this episode, including that fantastic paper by Morton Ann Gernsbacher and M. Remi Yergeau. And thanks to the Marc Sanders Foundation and the Templeton Foundation for their support of the show.