A recent study, published after a painstaking years-long effort, has claimed that findings in the social sciences in general, and psychology in particular, are quite difficult to reproduce and, therefore, impossible to accord scientific veracity to.
The Set Up
Well, the times had been tough of late for the social sciences. First, there was the Dutch psychologist, the numbers in whose study, it turns out, were made up. Then, there was the controversial study in the Journal of Personality and Social Psychology that seemed to support ESP. And then, there was the political science study that gauged the effects of gay canvassers on voters’ behaviour.
All this spurred on the good people at the Centre for Open Science in Charlettosville to engage in a project appropriately titled the Reproducability Project.
They took apart 100 studies published in three leading psychology journals and came to conclusions that would make even the most spirited defenders of social science squirm in their seats.
So, How Bad Were The Prior Psychology Studies?
Well, a score of 36 per cent isn’t bad when you consider….okay, it’s bad. These are professional researchers whose studies we’re talking about. That’s a horrible score. Only 36 per cent of the results of the studies actually held up in the reproducability study.
Yes, There Are Many, But How About Three Examples?
- A 2008 paper that had concluded that people are more likely to cheat after reading an essay that had argued that behaviour was predermined by environmental factors. The reproducability study found a much, much weaker correlation than the original study. And bad research is the gift that keeps on taking: it was cited 341 times in other journals.
- Another 2008 study had found that the severity in people’s moral judgements had changed after they thought about the idea of physical cleanliness. The researchers seemed to have proved a link between the two ideas. When the study was redone, the correlation wasn’t nearly as strong.This original study had been cited 17 times in other journals.
- Another psychology Study of the same year had suggested that lonely humans tend to create human connections to inanimate objects (think Tom Hank’s and his Wilson in Cast Away.) The redone study found that the researchers had manipulated the respondents into displaying behaviour that seemed to corroborate with their thesis. This original study was cited 13 times in other journals.
So What Does This Mean?
Well, the verdict isn’t out as yet. It’s not as if this is the first time that previous experiments have been redone. And the methods employed by the newer researchers might also be compromised, specially if they had set out to prove that the original conclusions were wrong. Psychology studies are Twisting the data to fit the model and not the other way round.
This does cast a doubt on all the increasing number of Americans seeking therapy. Though the study hasn’t been tough of psychiatrists, the psychologists have a lot of thinking and introspection to do, specially if they have been using the results of leading peer-reviewed journals in their work with patients.