The FT’s analysis of autism and ADHD prevalence is deeply worrying

Posted on

I have, for a while, thought that John Burn-Murdoch, who works for the Financial Times, is one of the best statistical journalists in the business. I had, in fact, grown accustomed to accepting his data. But I will never do so again. Yesterday, the Financial Times published an article by him in which he noted:

Between 2008 and 2023, the share of students at UK universities who reported having a disability doubled from 8 to 16 per cent. At the elite institutions of Oxford and Cambridge, far from bucking the trend, the increase was even steeper, with rates quadrupling from 5 per cent to around 20 per cent.

He added:

This trend is not unique to Britain, as disability services and resources in education come under increasing strain from rapidly expanding criteria. In the US, research by Rose Horowitch for the Atlantic magazine found that 38 per cent of undergraduates at Stanford this year are registered as having a disability, as are 21 per cent at Harvard — both up from 5 per cent in 2009.

And, to contextualise this, he said:

One could argue these figures tell a positive story of improved identification and care for young people who previously lacked help and support — and certainly there are elements of this in places. But deeper interrogation of the data paints a less rosy picture, particularly in the UK and US. It lays bare systems that benefit few and frequently fail those most in need.

In other words, his whole article was premised on the idea (admittedly, never explicitly stated) that there might be people claiming to have special needs who are exploiting these to secure personal advantage at cost to others, as this implies:

The additional assistance provided for students in these categories often includes extra time in exams and additional classroom support, and has been shown to significantly boost scores. There is no question that this is a huge help to most of the young people with less acute cases.

The implication that there might be cheating going on appears to pervade this whole article.

Saying so, Burn-Murdoch appears particularly fixated on those with level one autism, which only requires low levels of support, and those with so-called attention deficit hyperactivity disorder (ADHD). He seems unaware of the fact that you can have both conditions at once, which is then described as AuDHD. In fact, we can be sure that this is his concern because he noted this:

The bulk of the rise in special support for youngsters is cases of non-profound [sic: there is no such categorisation] autism spectrum disorder (ASD), attention deficit hyperactivity disorder (ADHD) plus anxiety and mental health [sic: this reference makes no sense at all because mental health is what we should aspire to], all of which have flexible diagnostic criteria. Where detailed data is available, we consistently see mild, not severe, cases driving the rise.

The hint that I take from this phrasing is that, in Burn-Murdoch's opinion, those involved should "just pull themselves together". This is a view typical of the right-wing reaction to these conditions, which, to be precise, identify people who process information and learn, think, and communicate in ways that are not the same as the majority of the population. Having either, or both, is not an indicator of mental ill health: it is simply a signifier of difference, which is something that the antisocial neoliberal world cannot tolerate.

This inference is, in my opinion, callous. The suggestion that people wth autism, ADHD or AuDHD should just pretend to be "normal" when they are not is what all people with the conditions know they have been forced to live with, but should not need to do so.

The technical term for their doing so is "masking", which is the process of pretending to be "normal" in the way the majority of the population is, when you are not. It takes considerable skill, massive observational ability, and a lot of energy to do well. It places all those who have to do it at a disadvantage in society precisely because those who are deemed normal refuse to recognise the needs of those who are different. It is one of the major reasons for these people's burnout and stress, which is why the conditions are so linked to anxiety and depression, and absence from the workforce. Burn-Murdoch appears to want to reinforce this stress, and at no point does he imply that it should be alleviated. Instead, he says:

The expansion of the definitions in this direction is far from cost free.

In the UK, the ballooning special education needs and disability (Send) caseload exacerbates a funding crisis in local councils; demand far outstrips both forecasts and resources. The results are predictably dire. As the number of more mild cases receiving support has climbed over the past decade, average funding per child (including the most severe cases) has fallen by a third in real terms: outcomes for those receiving support have deteriorated and councils describe a system on the brink.

What is Burn-Murdoch's aim? It is not to:

  • Argue for more provision.
  • Accuse the government of inappropriate policy funding choices.
  • Suggest how additional funding could be made available, as is obviously possible.

Instead, he seems to highlight the need to cut services to meet the austerity agenda by denying services to those who need them, even if they are not the most severe sufferers of these conditions. Those with real needs must, it seems, pay the price for Rachel Reeves' budget obsessions in John Burn-Murdoch's opinion.

And, to compound matters, he then alludes to support for Robert Kennedy Jr's position on this in the USA, saying:

There are also consequences far beyond education. The conspiracy-laden crusades by US health secretary Robert F Kennedy Jr and others to discredit vaccines, blaming them for rising autism rates, would have less success had diagnosis rates not been inflated by expanded diagnostic criteria.

The hint is obvious. He is implying that the crisis in support is being created by those needing support, and that the political consequences are their fault, too.  It is either that, or he thinks their conditions are made up, which he then seeks to prove as if he had not already dug a deep enough hole for himself.

First, he says this:

Careful analyses using consistent definitions show little to no change in underlying prevalence.

This may be true, but he ignores a multitude of relevant issues, which I will note below, highlighting that the prevalence can be wholly unrelated to the number of cases diagnosed.

Second, he then referred to this chart he had produced:

There are so many potential statistical flaws in the presentation of this data that it is almost impossible to know where to start, so I did the obvious thing and went and read the paper John Burn-Murdoch refers to as his baseline to suggest that:

  • There is no increase in ASD symptoms in society.
  • There is no increase in ADHD in society.
  • Diagnosis rates are above prevalence rates, implying there must be overdiagnosis, as appears to be his goal with this piece, because it is hard to work out what else he is saying when the inference of all that he has put together implies that one thing, as do these charts.

The Swedish paper he refers to was published in English and is available here, free to access.

Having read the paper, I think it is technically flawed in itself for a number of reasons.

Firstly, the paper, as first published, was riddled with errors, as the list of corrections to it since publication shows. You could say all of these are semantic. You could also say they imply a lack of rigour both by the authors and the journal that published their work.

Secondly, this is not world-leading science. It appears that I  was the 23rd person to download the paper.

Thirdly, there is an obvious flaw in the population sample used. The study appears to be about both autism spectrum disorder and ADHD diagnoses, but it is not.  That is because:

  • In the first instance, it relates to diagnoses in twins, and not the population at large. I should add that there are good reasons why twins are studied, and that is because they are not typical. The authors and Burn-Murdoch ignore this and compare data on general populations with data on a potentially atypical population. I think that is statistically flawed: very obviously, twins may not behave normally with regard to ASD and ADHD, most especially if there is a genetic element to it, as there can also be to twin births. There is no baseline data here, then.
  • Second, it turns out the study is not of people with autism and ADHD, but of parents' perceptions of whether their children have these conditions, which is something quite different. There are good reasons why this matters.
    • Many parents might have profound and deeply rooted societal reasons for not wishing to diagnose their children as being within these categories.
    • Many parents of children with level one autism or ADHD are themselves likely to have these conditions, but will be undiagnosed because most older people have never considered the possibility of this as an explanation for their own lived experiences in life. As a result, they may well see nothing unusual, as far as the rest of society is concerned, about their children's behaviour. There is, therefore, a massive risk of subconscious cognitive bias in the sample group, leading to under-reporting. This fundamentally flaws the research.
    • This study does not refer to actual diagnoses amongst those supposedly researched, who are the children themselves. It is a study of their parents.
    • As the paper itself notes, only 41% of the parents of twins who might have been surveyed agreed to take part in the study, and tellingly, the paper itself notes:

Non-responding parents were more likely to have twins with a clinical diagnosis of ASD, ADHD, and ASD or ADHD, which may make the estimates lower than if we had access to the entire population.

In other words, the authors themselevs knew their paper was flawed by the methodology used.

  • Thirdly, the sample sizes are small. As a result, they may not be suitable for extrapolation.
  • Fourthly, the authors of the paper admit that diagnosis rates are much higher than they find, saying, for example:

In Stockholm County, a recent report showed a prevalence of ASD diagnoses of 5.8 % in 13–17-year-old boys and 4.0 % in girls (Jablonska et al., 2023).

And

[I]n Sweden the prevalence has increased in all age groups, but most notably in teenagers, and it has been estimated that 10.5 % of all boys in Sweden and 6.0 % of girls in Sweden aged 10-17 years had a diagnosis of ADHD in 2022 (Socialstyrelsen, 2023).

I have deliberately noted the Swedish data they themselves cite to make the cohort as comparable as possible to those children they supposedly studied (but, it appears, did not, because they asked their parents instead).

I have more issues with the paper than these, but they illustrate my concerns.  If I had been peer reviewing this paper (and I have peer-reviewed a fair number of academic papers), I would have strongly advised against its publication because of all the flaws noted. That is just my opinion, but that is what peer review is all about.

But then let me turn to why John Burn-Murdoch's chart is also flawed. What he appears not to have realised is that he is making a category error. The paper he has chosen to support his hypothesis is not about how many people have been diagnosed with ASD or ADHD. Instead, it measures how many people's parents report a certain level of ASD or ADHD-like traits on questionnaires, which is something many parents will be ill-equipped to do for reasons I noted above. So the underlying paper, and Burn-Murdoch, compare two quite different things as if they are the same. Most especially, perceptions are not diagnoses because a diagnosis also depends on things the questionnaires used in this study could not fully capture because those being asked did not, in all likelihood, have the ability to undertake a diagnosis of whether the traits actally caused significant day-to-day impairment, and whether the difficulties showed up in more than one setting (home, school/work, social life). This would be, not least, because the parents involved had most likely not only not recognised their own children's behaviour but had also adapted to it, meaning they were most likely to be unaware of it or to see it as normal for reasons noted above. Comparing a symptom-based estimate with a diagnosis rate is, then, simply not statistically valid. In that case, it is not a viable basis for drawing conclusions. And yet, Burn-Murdoch drew them. 

So why might he have done that, given the other, very apparent bias in what he wrote, already noted above? There appear to be three possibilities.

First, all the article might do is confirm the profound misunderstandings of those who have not bothered to acquaint themselves with the realities of ASD, ADHD and AuDHD. In other words, John Burn-Murdoch had not done enough research to write about this issue as he did.  This is the kindest possible explanation for what he has done. It suggests that this FT article is just based on ignorance. I can honestly say I hope that is true, but I must also add I am not convinced.

Second, it is possible that John Burn-Murdoch brought his own confirmation bias (conscious or unconscious) to the process and then applied to the process of creating the article the approach usually described as top-down thinking, which is commonly used by those described as "normal" in society who seek to confirm their prior hypotheses by finding evidence to support the conclusions they have already reached. In other words, Burn-Murdoch believes there is overdiagnosis of ASD and ADHD, as many on the political right wing claim, and sought to prove the fact using whatever statistics he could find, about which he was not sufficiently open-minded as a result. I stress that this may be unconscious bias, but it appears to be bias nonetheless, and as a statistician, he should be aware of all those risks. If this suggestion is correct, then, given the sensitivity of this issue and its enormous significance to those who have these conditions, for this bias to influence an article in the Financial Times is deeply dangerous, not least because the whole basis of this analysis seems so profoundly statistically flawed to me.

Third, there is something else at work in the tone of this article which is potentially much more pernicious still, as there is also deeply implicit in much of the commentary from right-wing politicians on this issue, which is the suggestion that those with these conditions are somehow cheating, when they are in fact suffering ill-treatment from a society that refuses to recognise that they think in ways different from those who might be described as more normal. As evidence, I would offer the fact that at no point in the article is the possibility considered that diagnoses may still under-represent the true number of people in our communities who do have such conditions, even though the paper referred to suggests rates of prevalence below the current treatment rates. What is more, there is no discussion of whether the level of support provided is appropriate or not. There is no discussion, therefore, of why we should care about this issue. Instead, there is an attempt to at least underplay and maybe deny its existence for many who are receiving treatment using statistics that I think are fundamentally flawed, accompanied by the fairly overt suggestion that manipulation to secure societal advancement is taking place.

Fourth, Burn-Murdoch also appears fixated on the idea that these traits are more commonly found in fee-paying schools or elite unversities, as if this were evidence of gaming the system rather than evidence that parents in the schools in question can afford to pay for diagnoses, rather than wait for the extended periods that are usually involved in securing these if the NHS is relied upon, a fact he did not note. This also implies, to me, the existence of at least a subconscious cognitive bias, implying once again the manipulation of diagnosis when, in fact, a much more straightforward explanation for the observed phenomenon is available.

In my opinion, and I stress that this is all that this represents, this is not only a very poor statistical analysis that does not support the claims made, but it also shows a profound bias on the part of the Financial Times and John Burn-Murdoch, which I find deeply unsavoury. It will, inevitably, be used to foster prejudice against people who have real needs because they think in ways different from the majority in society, which difference carries the real risk that they will be prejudiced against, ostracised, and positively discriminated against within a society to which they could bring so much. That opportunity will be denied because they will be deemed to be too deviant from what, to use a statistical term, might be called the mean of normality, which is the aspirational goal that neoliberal, antisocial economics appears to have for us all. For that reason, I found this article deeply disturbing and dangerous.

I will be forwarding this post to the FT for comment. When doing so, I will reserve my right to have quoted their article in the length I have as being a public interest use of what they published.


Comments 

When commenting, please take note of this blog's comment policy, which is available here. Contravening this policy will result in comments being deleted before or after initial publication at the editor's sole discretion and without explanation being required or offered.


Thanks for reading this post.
You can share this post on social media of your choice by clicking these icons:

There are links to this blog's glossary in the above post that explain technical terms used in it. Follow them for more explanations.

You can subscribe to this blog's daily email here.

And if you would like to support this blog you can, here:

  • Richard Murphy

    Read more about me

  • Support This Site

    If you like what I do please support me on Ko-fi using credit or debit card or PayPal

  • Archives

  • Categories

  • Taxing wealth report 2024

  • Newsletter signup

    Get a daily email of my blog posts.

    Please wait...

    Thank you for sign up!

  • Podcast

  • Follow me

    LinkedIn

    LinkedIn

    Mastodon

    @RichardJMurphy

    BlueSky

    @richardjmurphy.bsky.social