It’s coming up to that time of year again (in the Northern Hemisphere, at least). The daylight creeps longer into the evening hours, leaves on tree and shrub begin to glow in clouds of verdant splendour, migrating birds return to seek climactic asylum in our precinct of the ecosystem…and the mood on college campuses (and in many high schools) becomes afflicted by a shared anxiety. Yes, we are approaching Examination Season.

Another sign of Examination Season is the emergence of news stories in the mainstream media concerning Examination Season. Many such stories relate to the latest scientific finding that is somehow relevant to students and their exams. I suppose one reason for this is that most jobbing scientists work on or near college campuses, so their awareness of Exam Season is high. But the sheer availability of a huge population with bespoke occupational stressors might make research just a little too easy sometimes.

Consider this story, widely reported in the media the other day: “Taking water into exams could boost grades“, the Daily Telegraph declared. In what way, you ask? Well, according to the Daily Mail:

…one theory is that information flows more freely between brain cells when they are well hydrated.

Yeah, that’s how it works. In fact, that’s why we have grade inflation. Because of all the water in students’ brains.

As behavioural science news goes, this story travelled widely across the world’s media. As well as the UK staples (such as BBC News, the Telegraph, and the Daily Mail), the study popped up in the Toronto Star, the Irish Independent, the Phillipines’ ABS-CBN, India’s Asia News International wire service as well as the Times of India, and Singapore’s Today on Sunday.

Based on the extent of media coverage, you might expect this story to reflect a significant contribution to our understanding of how bodily hydration impacts upon cognitive acuity. I’d be guessing that we had a controlled trial here, where students were given graduated doses of water and compared to age-matched controls of similar academic abilities, all within the norms of a blinding protocol that prevented the researchers from inadvertently biasing the behaviour of their study participants.

After all, it’s not every day that psychologists discover a way of boosting measured intelligence (which is one important, albeit arbitrary, implication of enhanced test-taking proficiency).

But sadly we didn’t have any of that. What we had instead was a survey. Of 447 students. Basically, the researchers asked the students whether or not they brought water into their exams, and then looked at their exam results.

By way of sophistication, the researchers sought to statistically control for general academic ability. In other words, they factored marks gained in other assessments (such as term papers, projects, and similar activities usually described as “course work”) into their number-crunching. This was intended to control for the possibility that the students who brought water into exams were simply smarter to begin with.

However, this is an incomplete control. It wrongly assumes that both course work and examinations involve general academic ability and nothing else. But the two are not so simply comparable. The most obvious difference is that traditional written examinations are acutely time-constrained, with achievement measured during a specific time-window of performance.

Course work, on the other hand, is loosely time-constrained. Some students get good grades for essays and term papers that are written in a timely and efficient manner; others get good grades for essays that take hundreds of hours of inefficiently allocated time (you know, pulling all-nighters, acquiring squatter’s rights to a seat in the library, losing contact with friends and family, developing osteomalacia, etc.).

By using course-work performance as a statistical control for academic ability, the researchers are not necessarily capturing the students’ academic efficiency. And if “efficiency” is associated with things like remembering to bring water with you to the exam hall, then this would explain a statistical association between water habits and exam performance.

Yes, good time-efficiency might help you with course-work as well. But, because course work lacks the sheer against-the-clock speed-test dimension of tests taken in exam halls, its contribution will be more diffuse. And that difference in contribution is all it takes for it to fail as a statistical control.

In short, scientists may have lots of laboratory evidence about the benefit of water for mental acuity, but this particular study doesn’t demonstrate it. The researchers here didn’t even establish whether the students actually drank the water they had, just that they brought it with them.

My guess is that a similar study design would show that students who turn up early for their exams will also be found to perform better. As will students who remember to bring a spare pen, some tissues, a packet of Band Aids, a map, their house keys, and — I don’t know —  maybe even a banana into the exam with them.

Hmm. I’m now thinking it is only a matter of time before somebody actually conducts a study on that banana theory.

Check the newspapers this time next year to find out.

Brian Hughes

Brian Hughes is an academic psychologist and university professor in Galway, Ireland, specialising in stress, health, and the application of psychology to social issues. He writes widely on the psychology of empiricism and of empirically disputable claims, especially as they pertain to science, health, medicine, and politics.