One of the work outputs on the TraD project is is to deliver a course in research data management for Psychology postgradate students at University of East London. We’ve therefore been keenly following an academic scandal directly affecting the world of social psychology involving the Dutch psychologists Diederik Stapel (falsifying data) and Dirk Smeesters (massaging data). Similar scandals and retractions in the field have also involved Lawrence Sanna (2012) Marc Hauser (fabricating data 2010) and Karen Ruggiero (2001).
Brain storm: Social Psychology Theory / Distributed Cognition / CSCW by Rob Enslin
Dutch investigators have released their final report into the case of Stapel from Tilburg University, entitled: Flawed science: The fraudulent research practices of social psychologist Diederik Stapel.
Its findings reveal that Stapel fabricated data in 55 articles and book chapters. So far, 31 of those published papers have been retracted — three others have expressions of concern — although more might follow. In addition, 10 dissertations by students Stapel supervised were found to contain fraudulent data and this is what should be brought to the attention of our psychology Postgrad students here at UEL.
The final report makes for sober reading (Stapel personally taught his department’s scientific ethics course, for example) and a damning assessment of the discipline itself referring to “a failure to meet normal standards of methodology. [bringing] into the spotlight a research culture in which… sloppy science, alongside out-and-out fraud, was able to remain undetected for so long.” P. 5
The report has highlighted a number of issues with regards to research data handling, standards and attitudes and which we are likely to cover in our course. More points from the report:
- “The Stapel group had no protocols for, for example, the collection of data (including standards for questionnaires) or research reports. The PhD students in Mr Stapel’s group were not familiarized with fixed and clear standards. (Stapel) underscored the lack of (fixed and clear) standards; but that too was far from a local issue, according to Mr Stapel.” P. 42
We cannot be certain if as Stapel says this is not a local issue but one that crosses social psychology as a whole but how research data is collected and standards applied to that data (as well as taught) should be something that is paramount in the teaching of postgraduates. The report again:
- “The doctoral examination board must form a clear impression of the way in which research data has been collected.” P. 57
The importance of research data verification and replication of findings from research data will be something we shall likely emphasize to students studying for doctorates in our course.
- “Research data that underlie psychology publications must remain archived and be made available on request to other scientific practitioners. This not only applies to the dataset ultimately used for the analysis, but also the raw laboratory data and all the relevant research material, including completed questionnaires, audio and video recordings, etc. It is recommended that a system be applied whereby on completion of the experiment, the protocols and data used are stored in such a way that they can no longer be modified. It must be clear who is responsible for the storage of and access to the data. The publications must indicate where the raw data is located and how it has been made permanently accessible. It must always remain possible for the conclusions to be traced back to the original data. Journals should only accept articles if the data concerned has been made accessible in this way.” P. 58
There is not much we can add to this set of recommendations; whether they can or will be implemented is as yet unclear particularly as as far back as 2002 similar recommendations were made following the Ruggiero case. But there seems to be a desire to change and the way data is managed in this field is something that unbeknownst to us is something we are now a part of.
As Uri Simonsohn, the researcher who flagged up questionable data in studies by social psychologist Dirk Smeesters, has said: “We in psychology are actually trying to fix things… It would be ironic if that led to the perception that we are less credible than other sciences are. My hope is that five years from now, other sciences will look to psychology as an example of proper reporting of scientific research.”
We hope that the psychology course in research data management we will be running at UEL will be a part of this hoped-for progress and will be seen as an example of good practice for future psychologists.