Last week, the news agency Agence France-Presse (AFP) issued a press release with the title, “Rightwing candidates are better looking, says study”. The item described a study conducted by Swedish and Finnish economists in which 2,500 non-Finns rated 1,357 Finnish election candidates for attractiveness. The authors identified two main conclusions. Firstly, right-wing candidates were rated as being more attractive than left-wing candidates. And secondly, candidates rated as more attractive (who were mainly right-wing) were found to have received more votes. The press release was picked up and reported by a wide range of news outlets, including the Independent (London), the Daily Mail, the Wall Street Journal, Discovery News, the Albuquerque Express, The National (Abu Dhabi), the Times of Malta, the Ottawa Sun, and many more. It’s too bad then that: (a) the study has not been published in a peer-reviewed academic journal; and (b) its main findings are almost certainly unreliable.
In fairness, some of the newspapers carrying the story (the Independent, for example) stated explicitly that the research was not yet published in a peer-reviewed outlet, and this point was also included in the AFP press release. However, a report on the study itself was provided to the German Institute for the Study of Labour (IZA), and publication of a press release was initiated by someone (presumably either the IZA or the authors themselves) prior to peer-review. And the report was made available to the public to download from the IZA website. So although it is not peer-reviewed, it is for all intents and purposes published.
The main limitation on the study methodology relates to its sampling procedures. Data were collected by posting a survey online and then hustling for respondents in various ways. Some university students were invited to complete the survey by means of classroom announcements. Members of the public were solicited using newspapers and magazine advertisements, and through notices published on blogs. Despite running to 34 pages, the IZA report does not tell us which universities, newspapers, magazines, or blogs were involved. In fact, the report doesn’t give us much information at all about respondent characteristics (such as age or gender), although it we can make out that, in the main, respondents were located in two countries (Sweden and the United States).
In terms of data validity, the procedures contained no method of limiting multiple participation (as such, the same respondent could have completed the survey multiple times). Further, there was no measure for ensuring the accuracy of whatever personal details were provided. So although we have data which have been interpretated as indicating that some Finnish electoral candidates are better-looking than others, we don’t really know who actually provided these data. Thus we don’t really know to whom the findings can be generalized.
The data themselves are based on fairly simplistic ratings, with respondents invited to submit scores for beauty on 5-point scales. Ultimately, average scores for right-wing and left-wing candidates were compared using a t-test, even though the data are likely to have violated the statistical assumptions of t-test analysis (including assumptions that the data being analyzed will be normally distributed, that there will be equivalent statistical variance across the groups being compared, and that the data will have been gathered using random sampling).
Interpretation of the findings is compromised by a failure to adequately control for potential confounds. One simple example is candidate age. Given that attractiveness ratings are likely to conform to common cultural standards of “physical beauty” (itself a highly relativistic concept), there is a strong likelihood that older candidates would be rated as less attractive than younger ones, especially if they are female. Therefore, right-wing vs. left-wing comparisons can only be meaningful if candidates on both wings are equivalent in age (or if age is controlled for statistically). For example, if right-wing candidates are younger than left-wing ones, then this might account for differences in attractiveness ratings. Whether there are such age differences will be related to idiosyncratic political histories (such as whether right- or left-wingers have been in power in the target country for longer).
One hint that candidate age might have affected the IZA study is that the data showed different patterns for different subcategories of politician. With female candidates, the right-left difference in attractiveness was not statistically significant for candidates who had previously held office. Assuming that candidates who had been elected in previous elections are likely to be older on average than new ones, this pattern of findings is consistent with the idea that age is a confounding variable. In other words, right-wing politicians were not rated as more attractive because the were right-wing, they were rated as more attractive because they were younger.
Similarly, when considering the data on electoral success, we need to take account of the fact that (as well as candidate attractiveness) there are political reasons for election results. These political reasons may well be correlated with candidate age. For example, in the IZA study, right-wing candidates may have been both younger and successful because of political factors surrounding the election they were running in.
One such factor would be if the electorate were particularly inclined to vote for right-wing candidates because the available left-wing party had already spent a long period in office (in other words, the electorate felt it was time for a change). This is precisely what had happened in the present case. The IZA study covered the 2003 Finnish Parliamentary Election, an election that saw a large swing to the right and the removal from office of the left-wing Social Democrats who had been in power since 1995. Again this is consistent with the interpretation that age is confounding the study findings: attractive (i.e., younger) candidates received more votes because they were right-wing, rather than the other way around.
In the AFP press release, the authors do not ponder such chicken-and-egg banalities but instead conclude that beauty in politicians must reveal deep-seated psychological attributes of relevance to the electorate. They argue that “the left perhaps traditionally has used a more rational approach [when conducting politics]. The right meanwhile has been more conscious of the importance of looks”. In other words, they argue that being right-wing makes candidates focus on their appearance and other superficial matters. In the Independent (see above) they offer an alternative interpretation, namely, the exact opposite argument: “One possible explanation is that people who are seen or consider themselves beautiful tend to be more anti-egalitarian and rightwing“. In other words, this time the argument is that being attractive precipitates right-wing attitudes. However, given that the researchers collected no data on the beliefs and attitudes of politicians, they have no basis on which to make either assertion. Some would say that these speculations amount to little more than psycho-babble.
In summary, while the researchers went to great trouble to set up their online survey (which required the sourcing of many hundreds of candidate photographs), their study does not support the conclusions that are presented. The sampling is rough and ready, the data are not clearly amenable to the statistical analyses used, and the role of extraneous variables is not accounted for at all.
As mentioned, some newspapers who carried the story stated explicitly that the study was not yet published in a peer-reviewed journal. However, I wonder if this type of disclaimer is sufficient. Firstly, how many readers would be familiar enough with the peer-review process to understand the actual meaning of that disclaimer? And secondly, what are readers to take from the fact that the news item is being published anyway? Surely they will assume that its publication in news outlets implies that the study has been somehow validated? After all, what would be the point of issuing a press release (or running a story) if readers were intended to ignore its content? The fact is that these findings were relayed as news in the AFP press release, and were duly treated as such in all outlets that carried them. As far as I can make out, none of these outlets raised any queries about the study’s validity.
In this case, readers were presented with ‘scientific findings’ that were neither ‘scientific’ nor ‘findings’.
This type of thing just doesn’t look good.
Brian Hughes is an academic psychologist and university professor in Galway, Ireland, specialising in stress, health, and the application of psychology to social issues. He writes widely on the psychology of empiricism and of empirically disputable claims, especially as they pertain to science, health, medicine, and politics.