Category: Alternative/Complementary Medicine

My letter to the BMJ regarding their lax editorial approach to bogus therapy paper

Here it is. I think it is pretty self-explanatory (nonetheless, I have added some additional context below the fold):

Date: Sep 11, 2019
To: <Fiona Godlee>, Editor in Chief, BMJ

RE: BMJ’s scientifically and ethically indefensible decision about Bristol’s Lightning Process study

Dear Dr Godlee,

First of all, thank you for the work that you do as editor in chief of the BMJ. I do not at all mean this as faint praise. While my colleagues and I have expressed our concerns, I believe it is important to note that editing a journal is not always easy. Editors play a crucial part in the production line of science and your own contributions, and those of your editorial colleagues, need to be acknowledged.

Secondly, I feel obliged to add my voice to those others who have written to you about the paper “Clinical and cost-effectiveness of the Lightning Process in addition to specialist medical care for paediatric chronic fatigue syndrome: randomised controlled trial” and, particularly, the associated correction that was published.

You will be familiar with the details of our concerns as outlined in previous emails.

From my own perspective, I have been investigating the issues around controversial therapies and human factors (i.e., error risks) in research for nearly two decades. You will be aware that expectancy effects and confirmation biases are very problematic in clinical therapy research in general. They create particular chaos with therapies that are controversial to begin with.

The so-called ‘Lightning Process’ is undoubtedly one such therapy. In fact, it comprises a number of modalities that are normally classified as pseudosciences. It is based largely on so-called Neuro-Linguistic Programming (NLP), which is a completely discredited practice. The creator of the ‘Lightning Process’ is an advocate and provider of several pseudoscientific therapies (e.g., cranial osteopathy; applied kinesiology; hypnotherapy). All told, there is nothing to suggest that the ‘Lightning Process’ is a promising clinical modality. It has no scientific plausibility; it exists because commercially-minded providers of pseudoscientific treatments have successfully identified a market for it. In that regard, it occupies the same space as, say, crystal therapy.

In offering that description, I do not intend to be pejorative. I merely wish to point out that any hypothesis that the ‘Lightning Process’ is effective for any condition constitutes an “extraordinary claim.” By all means should extraordinary claims be tested. Studies of crystal therapy, for example, should be conducted. However, you will appreciate that in science, “extraordinary claims require extraordinary evidence.” Research purporting to establish the efficacy of the ‘Lightning Process’ needs to exhibit a truly compelling evidentiary standard. The claims made inherently challenge our mainstream understanding of human biology. If the ‘Lightning Process’ is in fact shown to work, we will need to re-write our physiology and neurology textbooks.

On the face of it, the evidence contained in this particular paper is far from extraordinary. In fact, as per your own published correction, the study bypassed the normal requirement for prospective registration. This immediately raises the red flag of reporting bias. The evidence accrued from the paper cannot be relied upon because the authors were able to select to report it, without pre-registering their intentions.

In short, confirmation bias was not prevented. Your request to the authors that they tell you whether or not they were biased is very insufficient.

The data produced in light of this procedural and methodological lapse by no means meet the standard of “extraordinary evidence”.

As I understand it, this paper would not have been published had you known about its irregularity at the time of initial review. Therefore it should not be published now. Therefore it should be retracted.

A serious problem with not retracting this paper is that its existence will strongly encourage other researchers to play fast and loose with pre-registration, in the knowledge that their findings can still be published in a BMJ journal. The paper will stand as a high-profile exemplar of ambiguous editorial commitment to scientific rigour.

Kind regards,

Professor Brian Hughes, PhD, FPsSI
School of Psychology
National University of Ireland, Galway

For context, my email is part of a now lengthy correspondence following an Open Letter to the BMJ arising from the correction of the ‘Lightning Process’ paper. I was one of several international scientists to co-sign the original letter, which called for the paper to be properly retracted. Afterwards, several individual signatories provided their personal perspectives as I have done above.

It is worth noting that I deliberately focused on the matter of confirmation bias, as it reflected a point that I felt would benefit from greater emphasis and about which I feel well qualified to comment. 

However, there are a number of other problematic issues relating to this paper. These include the very poor methodology employed in the study, where (again) self-report methods are relied upon in the absence of adequate blinding in a clinical trial. This makes the study wholly unreliable, and raises questions about the quality of peer review used by the journal.

In short, why exactly did the journal fail to pick up on such an elementary methodological weakness?

People will have varying opinions about why this happens so frequently. I have written before about how some fields come to be characterised by mass incestuous deference to eminent researchers and their acolytes, and about how unpoliced pandering frequently diminishes the quality of science by neutering peer review. 

I suspect I will need to write about that issue again…

…and again…

and again.

For those interested, you can find out more information about the so-called ‘Lightning Process’ here

Um Starbucks, could you not?

The Starbucks I go to is now selling magic beans. Well, matcha tea lattes actually, which are like magic beans in the sense that they possess special powers:

Detox the body

High in antioxidants

Helps the immune system

Burns calories

I was particularly intrigued at the last claim. This latte actually “burns calories”.

Which is strange, because a venti serving of said magic tea latte — sorry, matcha tea latte — contains 316 calories all on its own, making it the most calorie-laden tea drink on the menu:

This is pretty much a similar number of calories as in one of those super-sized chocolate chip cookies they sell.

Hmm. Do the chocolate cookies also “burn” calories?

Obviously this is all ridiculous. When you drink a venti matcha tea latte, you consume calories, you don’t burn them. You consume as many calories as you would if you ate a chocolate biscuit.

Physical activity burns calories. Some other things (like stress) can also burn calories, but not always in a good way.

You can’t burn calories by consuming them.

But I guess anything that claims to perform “detox” is bound to be bunkum.

‘Rethinking Psychology’ is now available

Alright, there really is no humble way of putting this. My new book [*blush*], having been trailed as “imminent” for several months, is now officially available. In all good booksellers, as they say (and they actually do say this).

I’ll be having an initial launch event in Galway in late April (details to follow). But in the meantime, here are all the formal bits and pieces you need to know…

rethinking

Imprint: 2016
Rethinking Psychology: Good Science, Bad Science, Pseudoscience
Author: Brian M. Hughes
Publisher: Palgrave, London

ISBN-10: 1137303948
ISBN-13: 978-1137303943

Click here to view on Palgrave
Click here to view on Amazon.co.uk
Click here to view on Amazon.com
Click here to view on Amazon.in
Click here to view on Amazon.co.jp
Click here to view on Barnes & Noble
Click here to view on Book Depository
Click here to view on uRead (India)
Click here to view on Waterstones
Click here to view on WHSmith

From the cover: Psychology is one of the most popular subjects in universities across the world, offering unique insights into the human condition. However, its very popularity threatens to undermine its value as a discipline, and it often attracts those who lack scientific rigour. Taking a fresh look at common practices and pitfalls, Brian Hughes examines the relationship between psychology, science and pseudoscience, and explores the biases impeding many psychologists from being truly rigorous.

Brian Hughes has written an important and engaging book exploring the relationships between science, pseudoscience, and psychology. He argues persuasively that psychology itself can properly be considered to be a true science but one that is marred within by pockets of pseudoscience. This book should be read by anyone with a serious interest in the subject.” — Professor Christopher French, Goldsmiths, University of London

“Hughes provides a timely and comprehensive reminder of the critical role of science in both academic and professional applications of psychology. It covers an impressive breadth of topics with incisive clarity and illustrates clearly the integral role of scientific approaches to understanding psychological phenomena.”Dr David Hevey, Trinity College, Dublin


 

Contents

PART I PSYCHOLOGY AND PSEUDOSCIENCE IN THEORY
Chapter 1 What is Science and Why is it Useful?
Chapter 2 What is Pseudoscience and Why is it Popular?
Chapter 3 The Scientific Nature of Psychology
Chapter 4 The Scientific Nature of Psychology
PART II PSYCHOLOGY AND PSEUDOSCIENCE IN PRACTICE
Chapter 5 Examples from the Fringes: From Healing the Mind to Reading the Body
Chapter 6 Examples from the Mainstream: Biological Reductionism as Worldview
Chapter 7 Examples from the Mainstream: What Some People Say about What They Think They Think
PART III PSYCHOLOGY AND PSEUDOSCIENCE IN CONTEXT
Chapter 8 Biases and Subjectivism in Psychology
Chapter 9 Religion, Optimism and their Place in Psychology
Chapter 10 Psychologists at the Threshold: Why Should We Care?

 

%d bloggers like this: