The scientific method is truly amazing. Not only can it be applied to such clichéd domains as physics, chemistry, biology, flying people to the moon, curing cancer, adding ears to the backs of mice, cloning sheep, and running tarantulas through MRI scanners, it can also be used to resolve age-old philosophical conundrums like “How do you know that a liar isn’t lying when he tells you he’s a liar?” And boy does this open up a world of exciting scientific possibilities. For example, according to the lead story in today’s (UK) Independent, researchers have now exploited their ingenuity to establish that the people of Britain are “facing a boom in dishonesty“. More than that: the British people are exhibiting “a dramatic decline in private integrity“. It’s all over the front page. It’s the main headline. It’s tagged as an “EXCLUSIVE“. In other words, it is REALLY BIG NEWS.

I think I will always remember where I was when I heard that the people of Britain were facing a boom in dishonesty. If 9/11 and the Challenger disaster were flashbulb memories, well this revelation is basically the supernova SN 2006gy of all human recollections.

Unfortunately, the underlying research from which the evidence for this extravagant scoop is drawn is beset by a few problems. But just a few. Things like the method, for example. And the data. And how the data were analysed. And interpreted. Just piddly stuff like that.

So how do we know that the British people are, to use the Independent’s phrasing, “becoming less honest“? Well, researchers at the University of Essex are reporting that a sample of Britons have scored lower on a standardized integrity questionnaire in 2011 than a different sample did ten years previously. That doesn’t seem to me to be that surprising. After all, when you run surveys across large-ish groups of people, it would be something of a mathematical fluke if the results always came out the same. So, then, question: how much of a difference was there? Well, over to the University of Essex press office:

A decade ago, seven out of ten people said an affair could never be justified. But by 2011 that figure had dropped to around half.

So, we’re going from seven-out-of-ten to five-out-of-ten on this one. That might be a big difference. But a difference in what? Does this issue really reveal a crisis of “dishonesty“? Or could it reflect a healthy change in attitudes to relationships (perhaps a diminution in the type of social conservatism that makes absolutist demands on people to remain in couples no matter what awful circumstances might arise)? A different interpretation here could be that the British people are becoming more open-minded. This would give us a crisis of open-mindedness, and we wouldn’t want that now, would we?

Okay. Anything else?

The proportion who said picking up money found in the street was never justified dropped from almost four out of 10 in 2000 to fewer than two in ten in 2011.

Oh. My. God. People are more likely to “pick up money found in the street” during the worst recession in living memory than during the peak of an economic boom. Well, I never. Of course let us note here that even during the halcyon days of 2000, the majority (6 out of ten) were quite okay with this type of behaviour. It’s just that that majority has increased a little in 2011.

But really, is “picking up money found in the street” actually indicative of dishonesty? What about finders keepers, and all that? I guess it depends on how much money we’re talking about. Unfortunately, the survey item used by the researchers (i.e., “Keeping money you found on the street“) doesn’t specify. It could just be 20p. After all, aren’t you much more likely to find a tiny amount of money lying on the street than you are to find a large amount? Maybe people are just factoring in the base-rate.

Okay. What else have you got? Well, to cut a long story short, here’s the lot (taken from the original research report as published by the university):

In summary, comparing 2011 with 2000, we can see the following:

• People nowadays seem more tolerant of sleeping around (under-age is fine) and dope smoking. But for me, these behaviours do not indicate dishonesty. If anything, they indicate a level of broad-mindedness.
• Secondly, people are more tolerant of keeping money they find on the street. But, as explained above, this is unsurprising and/or vague.
• Thirdly, people appear a bit more tolerant of failing to report accidental damage to another person’s car, and of drunk driving. But in both cases the difference from 2000 to 2011 is extremely small, and in both cases a substantial majority of the population remain intolerant of these acts.
• Ditto for buying stolen goods.
• Fourthly, there is virtually no difference in tolerance for lying (which is slightly more tolerated) or for dropping litter (which, actually, is slightly less tolerated).
• What else can we see? Well, the British people appear to be substantially less tolerant of social welfare cheats today than they were ten years ago. (Presumably not a crisis.)

Now I want you to just look at that bar chart again, except this time stare at it while repeating to yourself the conclusion of that front-page news story in the Independent: BRITAIN IS “FACING A BOOM IN DISHONESTY” AND “A DRAMATIC DECLINE OF PRIVATE INTEGRITY“. Hmmm. If by “boom” you mean “trivial change” and by “dramatic” you mean “non-existent” then I’m right there with you. All the way.

Two more quick points. Nowhere — and I mean nowhere — in the original research report do the investigators even attempt to run their figures through an elementary statistical test. Nowhere. This means that we cannot tell if any of the fluctuations observed were any more than random. In other words, we do not know if the data returned in 2011 and 2000 were statistically significantly different in any way.

And secondly, studies of this nature — which purport to compare public attitudes at two separate time-points — will depend heavily on the representativeness and comparability of the two samples used to provide the data. So how were the data collected? Well, the data were collected using online surveys. Do remember to cast your mind back to the year 2000. Remember what online surveys were like then? That’s right. Obscure. Only a tiny minority of the population could be reached this way.

But I think that the absolutely biggest problem with this story is the fundamental conundrum that I mentioned at the outset: “How do you know that a liar isn’t lying when he tells you he’s a liar?” Because actually, the researchers have not resolved this conundrum at all. And it undermines everything. Maybe we are observing a change in attitudes — but maybe people are becoming more honest, rather than less. Yes, that’s right, more honest.

Maybe, simply, people in 2011 are more honest about admitting to being tolerant of immoral behaviours (such as buying stolen goods and damaging other people’s cars) than their counterparts were in the year 2000. It could be that in 2000 people were just being sanctimonious gits, obsessed with looking moral in the eyes of mere survey-takers; while today, on the other hand, people are admirably up-front about their personal failings and don’t care who knows about it. A great improvement. Rejoice!

Spread the word. Britain is “facing a boom in HONESTY” and “a dramatic INCREASE in private integrity“.

You heard it here first. Honestly.

Brian Hughes

Brian Hughes is an academic psychologist and university professor in Galway, Ireland, specialising in stress, health, and the application of psychology to social issues. He writes widely on the psychology of empiricism and of empirically disputable claims, especially as they pertain to science, health, medicine, and politics.