I have recently been quoted in not just one, but two recent articles about the controversial psychotherapy, the Lightning Process (insert your own “lightning-never-strikes-twice” joke here).
Quite apart from my own contributions, both articles are well worth reading.
* * *
The first is from David Tuller, part of his “Trial By Error” series over at Virology Blog, in which he discusses a newly published systematic review of the Lightning Process.
The funny thing is, the review in question was written by none other than…the inventor of the Lightning Process!
Spoiler alert: the inventor of the Lighting Process found that the therapy he invented was indeed effective.
I have read the paper in question. Its findings are, shall we say, pretty weak.
The title says it is a “systematic review.” However, the paper does not contain a statistical synthesis of findings drawn from separate studies, as a formal “systematic review” actually should. Instead, it consists of a set of paragraphs describing ten surveys and four “non-survey” statistical studies of LP. Here is my summary of those non-survey studies:
- Study 1 is an unpublished study from 2013. It is posted on ResearchGate as a pre-print, but has never actually appeared in any journal. As it has therefore not been peer-reviewed, it benefits from no quality control. It is academically unsound and should not be included in such a review.
- Study 2 is another study by the author of Study 1, this time from 2014. This one also appears on ResearchGate but again in no journal. So, as before, it should not be included in this review. Like Study 1, it seems to have been listed on ResearchGate as a “pre-print” for the best part of a decade.
(From what I can make out, the author of both above papers has published just one formal journal article in their career, which they co-authored with…the inventor of the Lightning Process! Small world!) - Study 3 is a study of seven teenage girls and five teenage boys who rated their own pain after undergoing the Lightning Process. The sample in this study was minuscule, and there was no control group. And as it was published on a “health news” website, we can presume this one wasn’t peer-reviewed either!
- Study 4 is the only one actually eligible for review (making this whole exercise hardly a “review” at all), having actually been published in an academic journal. It’s the paper that reports on the SMILE study, memorably described by a patient group as “one of the worst examples of a clinical trial supposedly designed to assess the acceptability, effectiveness and safety of a treatment.” As well as being methodologically flawed, the SMILE study was ethically problematic. It bypassed the normal requirement for study registration, leading to public complaints from dozens of scientists. Despite the outcry, the journal’s editors chose not to retract the paper, even though had they known about the registration problem in advance, they would never have accepted it for publication. You can read all about this fiasco, again from David Tuller, here and here.
This so-called “systematic review” is surely one of the most atrocious academic papers that I have ever had the misfortune to read. It isn’t even a “systematic review”. Rather, it is a self-serving pseudostatistical jargon-filled waffle-fest, utterly untroubled by even the tiniest smidgen of scholarly objectivity. It is, in fact, deeply depressing.
This travesty has been accepted for publication in a journal called EXPLORE, which I have never heard of before. I can only imagine its peer-review process is extremely light-touch.
Be sure to go read David’s article here.
* * *
The second article is by Nina E. Steinkopf over at MElivet. She describes an application for ethical approval for a Lightning Process study received by the Regional Committees for Medical and Health Research Ethics (REK) in Trondheim, Norway. The applicants describe the Lightning Process as a “biopsychological” intervention.
In her article, Nina sets out a series of concerns relating to the proposed study, which are numerous and very troubling. You can read all about them here.
Nina reached out to me recently for advice on the matter. She quotes me as follows:
I do not describe the Lightning Process as a biopsychological intervention. To describe LP as “biopsychological” assumes that it is partly biological and partly psychological. This is incorrect. Both biology and psychology are evidence-based sciences. LP is clearly not.
A better description would be Alternative Medicine, or indeed Alternative Science. It is grounded on assumptions about the human body and mind that have no scientific basis (for example, the principles of Neurolinguistic Programming — scientifically discredited for many years). In that sense, LP is an unambiguous example of pseudoscience.
I recognise that even pseudoscientific interventions should be subjected to scientific scrutiny. However, there is an ethical question regarding the use of known pseudo-therapies in vulnerable groups, even for research purposes. In my view, LP presents a significant ethical challenge in this regard.
The REK will make its decision in the coming weeks. Here’s hoping common sense — and patient safety — prevails.
You can read the rest of Nina’s article here.
Brian Hughes is an academic psychologist and university professor in Galway, Ireland, specialising in stress, health, and the application of psychology to social issues. He writes widely on the psychology of empiricism and of empirically disputable claims, especially as they pertain to science, health, medicine, and politics.
