Friday, May 01, 2015

Flaws in the Peer Review Process

Scientific peer reviews are a 'sacred cow' ready to be slaughtered, says former editor of BMJ

The process is considered the gold standard of quality scientific research

The peer review process – long considered the gold standard of quality scientific research – is a “sacred cow” that should be slaughtered, the former editor of one of the country’s leading medical journals has said. 
Richard Smith, who edited the British Medical Journal for more than a decade, said there was no evidence that peer review was a good method of detecting errors and claimed that “most of what is published in journals is just plain wrong or nonsense”.
Research papers considered for scientific and medical journals undergo a process of scrutiny by experts before they can be published. Hundreds of thousands of new studies are published around the world every year, and the peer review process exists to ensure that readers can have confidence that published findings are scientifically sound.
But Dr Smith said pre-publication peer review was slow, expensive and, perhaps ironically, lacking in evidence that it actually works in its chief goal of spotting errors.

More here: 

Climate change emails between scientists reveal flaws in peer review

Read article here 
Scientists sometimes like to portray what they do as divorced from the everydayjealousies, rivalries and tribalism of human relationships. What makes science special is that data and results that can be replicated are what matters and the scientific truth will out in the end.
But a close reading of the emails hacked from the University of East Anglia in November exposes the real process of everyday science in lurid detail.
Many of the emails reveal strenuous efforts by the mainstream climate scientists to do what outside observers would regard as censoring their critics. And the correspondence raises awkward questions about the effectiveness of peer review – the supposed gold standard of scientific merit – and the operation of the UN's top climate body, the Intergovernmental Panel on Climate Change (IPCC).
The scientists involved disagree. They say they were engaged not in suppressing dissent but in upholding scientific standards by keeping bad science out of peer-reviewed journals. Either way, when passing judgment on papers that directly attack their own work, they were mired in conflicts of interest that would not be allowed in most professions.
The cornerstone of maintaining the quality of scientific papers is the peer review system. Under this, papers submitted to scientific journals are reviewed anonymously by experts in the field. Conducting reviews is seen as part of the job for academics, who are generally not paid for the work.
The papers are normally sent back to the authors for improvement and only published when the reviewers give their approval. But the system relies on trust, especially if editors send papers to ­reviewers whose own work is being criticised in the paper. It also relies on anonymity, so reviewers can give candid opinions.
Cracks in the system have been obvious for years. Yesterday it emerged that 14 leading researchers in a different field – stem cell research – have written an open letter to journal editors to highlight their dissatisfaction with the process. They allege that a small scientific clique is using peer review to block papers from other researchers.

No comments: