Lines are drawn
A quarter of a million Britons are believed to have myalgic encephalomyelitis (ME), the condition that sometimes overlaps with chronic fatigue syndrome (CFS). ME/CFS is a severe debilitating illness that renders patients either temporarily or perennially immobile. Sufferers will have often led very active lives before being struck down. After onset, they are forced to give up work or education. Many patients become bedridden for decades. People with ME/CFS experience a full syndrome’s worth of symptoms, dominated by ‘crashes’ of extreme unrelievable tiredness (formally referred to as ‘post-exertional malaise’), along with muscle weakness, digestive problems, pain, and hypersensitivity to light or sound.
The World Health Organisation classified ME/CFS as a neurological disease as far back as 1969. Since then, extensive research has shown that ME/CFS involves disruption of several bodily systems, including the nervous, immune, endocrine, cardiovascular, and gastrointestinal systems. Recent studies have revealed extensive cell-functioning impairment arising from irregularities in patients’ DNA.
Despite all the signs that ME/CFS is an organic disease, the precise biomechanics — and therefore, the most appropriate biomedical treatments — are still unclear. Some patients seem to do well with rintatolimod, an immunostimulant, but not enough to suggest it as a universal therapy.
But if you don’t have ME/CFS or know someone who has, chances are you’ve heard of it anyway. Most likely you will know that the condition is controversial. This is because, well, it is. From diagnostics to treatment to research, the world of ME/CFS is mired in never-ending acrimony. Some researchers claim to have been threatened with violence. A number have won humanitarian awards for standing up for their clinical views. Doctors defending medicine — from patients.
* * *
“All in the mind”
A recurring problem relates to how some doctors perceive the nature of ME/CFS, and the treatment decisions that flow from their assumptions. In Britain especially, the dominant clinical view is that ME/CFS is effectively a psychogenic disorder — one that is caused, or at least exacerbated, by patients’ thoughts, attitudes, and beliefs — and not a truly organic illness that involves irregular body chemistry, treatable using pharmacological or surgical intervention.
In other words, in Britain the ME/CFS world is dominated by psychologists, psychiatrists, and behavioural practitioners. Accordingly, the National Institute of Clinical Excellence (NICE) have adopted guidelines stating that ME/CFS should be treated using either cognitive behaviour therapy (CBT), graded exercise therapy (GET), or a combination of the two.
For years, patients have complained that these treatments are of no use. GET is especially disliked. For most people with ME/CFS, exercise leads to extreme pain and debilitation. The very thought of more physical activity is abhorrent. The fact that doctors prescribe exercise as therapy, and dismiss patients as malingerers when they refuse to comply, is endlessly horrifying. In the minds of some, it is little short of cruel and inhumane abuse.
Many UK clinicians are dogged in their defence of what has become known as the biopsychosocial model of ME/CFS. This is despite the physiological science that suggests they are wrong. Such is the root of the controversy: these doctors are sticking to their guns no matter what scientists — or patients — have to say.
British clinicians, you see, have done their own research, duly published in the best medical journals. They say their findings justify the use of CBT and GET — and nothing else — to treat ME/CFS. Unfortunately, their research serves to compound the controversy, rather than to calm it. The very fact these medics refuse to budge in the face of criticism makes the whole situation worse. The atmosphere has become truly toxic. Patients are haughtily dismissed as aggressive and unreasonable know-nothings.
It’s no wonder they are angry.
* * *
Keeping PACE
One of the biggest ever psychology studies to be conducted in a healthcare setting in Britain was the PACE Trial, a five-year investigation involving more than six hundred patients that set out to test the effectiveness of CBT and GET as treatments for ME/CFS. Since its inception in 2005, the PACE Trial has been plagued by problems, ranging from disputed diagnostics, to legal proceedings, to claims of dodgy data. In truth, the study’s a right mess.
Despite being extremely well resourced — and its main results appearing in The Lancet, no less — the PACE Trial contained a number of highly odd design features. For example, it relied heavily on self-report measures, but did not employ blinding. Any psychology undergraduate will tell you that self-report studies need to be blinded. Therefore — by design — the PACE Trial stands as a textbook example of bad research. (Literally, this year, it has appeared as such in a textbook.)
As a result of this flaw, we know that the PACE Trial findings must be untrustworthy. This is even before we see any data. The research was holed below the waterline from the very beginning.
As it happens, the data make for gruesome viewing. The biggest defects relate to outcome-switching. After the PACE Trial investigators began their research — presumably, once they crunched some numbers — they decided to change their definition of “recovery”. They did this by reducing the threshold of (self-reported) physical fitness required for patients to be deemed “well”. In short, they moved the goalposts.
This allowed them to report that one-in-five patients were classified as “recovered” when treated with CBT or GET, a lot more than would have been the case had they stuck to the original study protocol. The problem is that they forgot to do anything about the threshold they set for patients to be declared “sick” in the first place. Hilariously (in the ‘gallows humour’ sense), the result of their on-the-hoof adjustments was that their new threshold for “recovery” was now below the threshold for “sickness”.
Patients could be classified as “recovered” even if their scores for physical well-being declined. Somewhere in the region of one-in-eight patients fell into this surreal category. These patients were sick enough to take part. Their conditions deteriorated after therapy. But the investigators still presented their trajectories as evidence that the PACE Trial treatments had worked.
A lay person would be forgiven for calling this method “absurd”.
To say that the PACE Trial is an extremely weak study is to understate an obvious calamity. Such has been the resulting criticism from around the world that last year the Journal of Health Psychology was able to set aside an entire issue just to publish peer-reviewed articles tearing the PACE Trial to smithereens.
In Psychology in Crisis, I use the PACE Trial as an archetypal example of bad psychological science. It is truly emblematic of the chaos currently engulfing psychology. In recent years paranoia about psychology’s poor replication record has become endemic. Studies like PACE make psychology seem a truly dubious endeavour, and worsen the field’s already dodgy reputation. For scientifically-minded psychologists, it is the stuff of nightmares.
But the PACE Trial is not just a case study in flawed research design and iffy methods. It also highlights the way professional politics can intrude on the scientific process. It shows us how professional groups sometimes try to face down objective criticism, look the other way when ineptitude is exposed, and doggedly stick to their guns in order to avoid threatening their own interests.
These days we have a term to describe how repeated mantra-like talking-points can be employed to suppress legitimate and objective rebuttals. We call it “post-truth” argumentation. Such gamesmanship is not confined to traditional politics. Psychologists can do “post-truth” too.
The world of clinical disease management, especially that of ME/CFS, is just as likely to become embroiled in a culture war.
* * *
Wagons being circled
Last month, Reuters published an exclusive story reporting how an influential Cochrane review, which had originally concluded that GET is an effective treatment for ME/CFS, is now to be “temporarily withdrawn” in light of “feedback” received regarding its methods and analysis. The review, published in 2017, had attracted scathing criticisms for what were identified as serious methodological flaws.
For example:
- The sample of studies included was highly selective
- Positive outcomes were exaggerated
- An overall lack of significant treatment effects was either overlooked or misreported
- Studies relying on self-report measures, which are usually considered weak in research terms, were analysed as though they were just as strong as studies that used objective measures
- The reviewers decided to shift their analytic approach after they had started their work (in psychology, we might refer to this as “a pattern of behaviour”). Their adjustments ultimately enabled them to report maximum effects for GET. Deviating from pre-planned protocols is a huge risk to the validity of any review, and is contrary to Cochrane good practice guidelines.
Many observers also pointed out that the review’s authors included investigators involved in the now beleaguered PACE Trial, meaning that they were effectively involved in a process of marking their own homework.
These criticisms, along with many others, were formally submitted to Cochrane and are currently viewable on their website.
In explaining Cochrane’s decision, Reuters highlighted pressure from patient activists who object to ME/CFS being depicted as a psychological condition. Discernible in the Reuters report was the sound of professional wagons being circled. A former head of Britain’s Medical Research Council was quoted as complaining that withdrawal decisions should not be dictated by patient lobbying.
This week, the Guardian posted a ‘science’ podcast discussing the Cochrane review withdrawal. In the blurb, they perpetuated the (inaccurate) Reuters reporting line, embellishing it with their own reference to “verbal abuse”:
…researchers looking at efficacy of these therapies claim they are verbally abused. But now, it may have gone a step further. Reuters recently reported that ‘a respected science journal is to withdraw a much-cited review of evidence on an illness known as chronic fatigue syndrome amid fierce criticism and pressure from activists and patients.’
More strikingly — perhaps even suspiciously — the producers of the Guardian science podcast decided that this week’s guest should be none other than a colleague of one of the authors of the withdrawn Cochrane review, and — for good measure — himself an author of the maligned PACE trial. Not just marking his own homework anymore, but now — in defiance of expert criticism — defending how well it had been marked.
In post-truth terms, it all felt a bit like an episode of Fox & Friends.
Cochrane doesn’t withdraw reviews lightly. In reality, this review was withdrawn from the public record because for nearly two years independent researchers had highlighted a steady stream of specific scientific criticisms. The Editor in Chief of Cochrane stated clearly that “this is not about patient pressure.”
In other words, the methodological problems proved to be just too glaring. Someone who headed the Medical Research Council should really be able to see that, as should the producers of the Guardian’s science podcast.
Perhaps they are the ones who are succumbing to partisan pressures.
* * *
Friends of friends
It is worrying that review practices intended to be scientifically objective seem so susceptible to distortion by social network effects, cultural etiquette, and a prerogative to preserve colleagues’ professional reputations.
The hegemony of the cognitive approach to ME/CFS is clearly partisan. It is at variance with much of the scientific literature. The very fact that its status is disputed exposes as logically unwise any claim that there is only one side to the ME/CFS story. Its endurance results from professional politics, not scrupulous science.
It is eminence-based, rather than evidence-based, medicine.
It is also very much a British thing. The corresponding American authority — the National Academy of Medicine — has declared ME/CFS an organic condition requiring biomedical treatment. Its genetic and neurochemical signs are well recognised (although its precise physiology is not yet established), as are the damaging effects of exercise therapies of the kind promoted as standard in the UK. It is the British clinical professions whose establishments believe that ME/CFS is psychogenic, that its main dynamics are “all in the mind”.
Medical conditions are not supposed to vary by international border. An infection contracted in one country should be expressed the same anywhere, and biomedical treatments should have a uniform effect regardless of what nation you live in. That the UK is an outlier when it comes to ME/CFS simply makes no sense.
Except possibly in one respect. We should not see ME/CFS as resulting from mass cognitive hysteria among a quarter of a million UK citizens. Rather, we could see the biopsychosocial theory of ME/CFS as a grand sanctimonious delusion shared by a professional clique who, for circumstantial reasons, find themselves dominant in British behavioural healthcare.
The echo chamber in which reviewers review each other’s work, award each other’s grants, and line up as one other’s acolytes, suggests that little of this will change any time soon.
Bad science is bad enough when it is just science. In the case of ME/CFS, where flawed research materially damages the lives of hundreds of thousands of blameless people, it is nothing short of a scandal, about which the establishment should feel acute embarrassment and, ideally, shame.
* * *
Edited, 3/11/18: Description of podcast rephrased to clarify that its guest was a colleague of one of the Cochrane review authors, as well as an author of PACE trial
Brian Hughes is an academic psychologist and university professor in Galway, Ireland, specialising in stress, health, and the application of psychology to social issues. He writes widely on the psychology of empiricism and of empirically disputable claims, especially as they pertain to science, health, medicine, and politics.
