You know, there are times when I almost manage to convince myself that psychologists aren’t a bunch of too-stupid-to-make-a-go-of-it-in-a-real-science idiots, and that they might actually have some useful insights into the management of the symptoms of mental illness. And then they go and publish utter twaddle like this…
Some psychologists at the University of Making Stuff Up Toronto engaged in a ridiculous orgy of timewasting study to look at the effects of loneliness and social ostracisation. In the first of two “experiments” they divided some volunteers into two groups. One group was asked to remember a time when they had felt lonely and unpopular. The other group were asked to remember a time when they had felt accepted and welcomed. Both groups were then asked to estimate the temperature of the room they had been sitting in. On average the ones feeling lonely estimated the temperature of the room to be lower than those who had remembered feeling accepted. The scientists psychologists concluded from this that feeling lonely or isolated makes you feel physically cold.
Ok, so hands up: what’s wrong with this “experiment”?
Where to start? Well, for one thing, there’s no control group. I would suggest in the context of this “experiment” that a control group could have been created by asking a third group of people to remember an event that was neutral in terms of its position on the welcomed – isolated spectrum. The scientists psychologists didn’t, it would appear, do this. One possible reason they didn’t bother with the control group is that, given the fundamentally flawed design of the “experiment”, it wouldn’t have made any difference.
The subjective experience of temperature varies from individual to individual. Put me and my brother into a room that’s heated to, let’s say, 17° C and my brother will go round obsessively looking for draughts and trying to find a nice warm cardigan while I’ll be trying to open the windows and complaining about how hot it is. The “experiment” as carried out had no mechanism for establishing whether or not there was a disproportionate number of subjectively hot or subjectively cold people in either group. Like the lack of a control group, there was actually a way round this problem.
The solution would have been to introduce a second stage to the “experiment”. In this second stage each group of volunteers could have been asked to come back into the room, which would have been maintained at a constant temperature. However, the volunteers would be told that the temperature had been altered, although they wouldn’t be told if it had been supposedly increased or decreased. They would then be asked to recollect the opposite experience to that which they had recalled in the first stage of the “experiment”, and would again be asked to estimate the temperature of the room. By tracking the difference in estimates of temperature made by the same individuals in different emotional circumstances it would have been possible to identify what effect, if any, social exclusion has on perception of temperature.
If the majority of volunteers estimated a lower temperature when recollecting a negative event, the experimenters would have had some grounds for concluding that remembering feeling isolated makes you feel cold. If the majority of volunteers estimated a higher temperature, they’d have had some grounds for concluding the opposite. But, unlike with the one stage experiment, there would also have been a third possible outcome – that there would be no clear majority either way.
The scientists psychologists who designed the one stage experiment appear to have created a falsifiable hypothesis – their experiment might have shown that, contrary to expectations, people felt subjectively warmer when they remembered feeling isolated. But there is actually a deeper hypothesis underlying the experiment which they have not attempted to falsify – that recollecting a negative social experience will have an effect on the subjective experience of temperature. The two stage experiment I have proposed here would, by allowing a potential outcome in which the data showed no clear majority either way, have enabled the deeper hypothesis to be proven or disproven.
The scientists psychologists did recognise one potential flaw in the first “experiment”. By asking their volunteers to remember a positive or negative situation, they had introduced an uncontrollable variable into the experiment. The second “experiment” tried to avoid the problem by attempting to directly re-create negative and positive social experiences. The experimenters did this by designing a computer simulation of a ball throwing game. Some volunteers had the ball thrown to them lots of times (this was supposed to make them feel included), while others didn’t (this was supposed to make them feel excluded). All the volunteers were then asked to rate the desirability of hot and cold food and drink. The volunteers who had been excluded showed a distinct preference for hot food and drink when compared to their more popular counterparts.
Again there would seem to be noticeable problems with this experiment. There appears to be no mechanism for taking account of the general preferences of the volunteers. The hot items on the menu were coffee and soup – volunteers who didn’t like either of those would have gone for the Coke and crackers no matter how excluded they felt. The reverse is obviously also true.
Again there would seem to be a simple solution. The volunteers could have been asked to rate the desirability of the various items both before and after they had their positive or negative experiences. By tracking any changes in preference it would have been possible to prove not only whether or not negative social experiences stimulate a desire for hot food and drink, but also the deeper underlying hypothesis – that snack selection is based on social experience. Again the experimenters appear to have created a falsifiable hypothesis – excluded volunteers might have, contrary to expectations, gone for the cold food – but have left a more fundamental hypothesis unexamined.
There’s something that bugs me really quite profoundly about this. I am by no means a scientist. My highest ‘achievement’ in the sciences was a narrow fail at A-level Biology, but even I can point out the obvious structural flaws in these experiments. The University of Toronto paper is due to be published in the journal Psychological Science (now there’s an oxymoron for you…). Psychological science is apparently not a formal research journal, but is nonetheless peer reviewed. So here’s the thing. If a paper so obviously flawed that even an unqualified amateur can point out its problems passes an academic peer review process, how can psychology even begin to claim status as a science?
Things actually get worse with the conclusions that are drawn from the data. According to the press release, the experimenters are not satisfied with simply establishing that an experience of social exclusion makes volunteers select hot food and drink:
Their preference for warm food and drinks presumably resulted from physically feeling cold as a result of being excluded.
Even if I were to accept that the selection of hot snacks is entirely the result of experiencing social isolation (which I don’t), this is an entirely groundless presumption. Hot foods are widely regarded as comforting. This may result from the fact that those in need of comfort feel physically cold, but it may also result from the fact that individuals have learned to associate warmth with comfort, irrespective of their actual perception of temperature. Nothing in the “experiment” carried out by the University of Toronto enables a conclusion to be reached on this issue.
Things get even worse when other scientists psychologists start commenting on the study:
Dr Lesley Prince, a lecturer in psychology at Birmingham University, said: “This is very interesting, and shows there are physiological correlates to emotions.”
Now, this is precisely not what the study has shown. Even disregarding the methodological flaws in the experiments, and the dodgy conclusions they lead to, all the study has “shown” is that people who are socially isolated have a subjective sense that their environment is colder, and that this makes them feel like having a nice hot drink. A ‘physiological correlation’ would mean that the objectively measured body temperature of socially excluded people was lower than that of socially included people. An experiment of this nature wouldn’t have asked volunteers if they felt cold, it would have used a thermometer and proved they actually were colder.
And there we have it. This whole thing is a ridiculous farrago of poorly designed experiments giving rise to unreliable data which is then misrepresented in order to suggest that it supports at best dubious, and at worst frankly ludicrous, hypotheses. This is nothing more or less than a grotesque parody of the scientific method, and it’s been the same with every psychology “experiment” I’ve ever seen described.
And yet psychologists feel qualified to call into question the efficacy of psychiatric medications which have been proven to benefit patients in rigorous double-blind clinical trials. In their place they want to promote their own unproven and unprovable theories as having greater objective value in the treatment of mental illness.
So. Is it just me, or are psychologists really taking the piss?