Mark Lorch 

Accusations of fraud spur a revolution in scientific publishing

Mark Lorch: Three and a half centuries after the first science journal was published, post-publication peer review is shaking up the old system
  
  

Man typing on a computer keyboard
Journals need to get used to the idea that in future peer review of papers will take place after publication as well as before. Photograph: Getty Photograph: Getty

Scientific publishing may be on the brink of a revolution fought, in part, within the chemistry blogosphere. In the past few months it has been the scene of debate about whether the scientific publishing practices initiated by members of the Royal Society almost 350 years ago are still fit for purpose.

In 1665, when the first scientific journal rolled off the presses, it was the cutting edge of science communication. The driving force for Philosophical Transactions of the Royal Society (as it is still known) was the members' desire to share their findings while being assured that no one could fraudulently claim credit for their work.

To ensure quality, all articles underwent peer review – a process that is still used by the thousands of journals that fill academic libraries today. The process is simple: authors submit their findings to a journal and editors send them out to be checked by the authors' scientific peers. If those peers deem the science to be valid then the journal publishes the paper.

But now a system that was designed to share knowledge and stop scientific fraud may be suffering from the very same problems that the pioneering publishers at the Royal Society were trying to overcome, namely fraud and poor communication. By some estimates between 1% and 2% of papers now contain fabricated data and many more are just plain wrong. If papers have flaws then these might get discussed in tea rooms and conferences, but without an organised way of communicating these conversations, the wider world remains ignorant of problems.

There have been murmurs in scientific circles about these flaws in the publication system for some time. But the issue blew up earlier this year when bloggers started reporting examples of suspicious practices in otherwise reputable journals. At first it was just a report of a throwaway comment accidentally left in a paper's annex:

"Emma, please insert NMR data here! where are they? and for this compound, just make up an elemental analysis … "

Some interpreted this as an academic inciting his student to commit fraud. Or maybe it was just an instruction to carry out a particular experiment. It was certainly sloppy practice by all concerned, including the editors, peer reviewers and co-authors, all of whom should have spotted the errant sentence. But it is not damning evidence of anything worse than this.

However, the post opened the floodgates as bloggers received tip-offs about other, more questionable activities. Apparently photoshopped images were found in the journals ACS Nano and Nano letters. Seemingly doctored spectra came to the fore in a paper from Organic Letters. An alleged case of self-plagiarism by an eminent member of the chemistry community was re-examined.

The debate on the role of blogs and other forms of social media in "post-publication peer review" was now out in the open. While not everyone was happy with social media being used to "out" dubious practices, saying the situation had descended into a witch hunt, most were more supportive.

Ultimately the bloggers' actions were vindicated, to the extent that all the suspect papers have now been withdrawn and investigations into scientific fraud are under way. Nevertheless the debate was largely confined to the blogosphere – until a few weeks ago when one of the journals responsible for publishing the supposedly photoshopped data, ACS Nano, waded in with a highly critical editorial aimed squarely at bloggers. It was a naive attempt to squeeze the social media genie back into the bottle while simultaneously trying to curtail free speech.

The article by the journal's editorial board included "instructions" on how suspected fraud should be dealt with:

"When plagiarism or data manipulation is suspected, accusations should be reported directly to the journal … "

Then they criticised the use of pseudonyms by bloggers and commenters:

"We strongly suggest that such comments be made without the cloak of anonymity … "

This seemed ironic since a central tenet of the peer review process is the anonymity of the reviewer, the point being that a reviewer can write what they truly think of a paper without fear of making enemies or losing friends. Why then should those who critique the paper after publication not have the same privilege?

Finally ACS Nano's editorial board threw some crumbs to the bloggers:

"After we have made our decision, all are welcome to comment on it in any blog, even if they have different opinions; this is their privilege."

I thought free speech was a right.

Of course bloggers responded angrily to the editorial. But the spat did serve to highlight the need for an open and fair mechanism for recording and collating discussion of scientific results. After all, handling cases of suspected fraud on a handful of blogs and Twitter threads is hardly ideal, and the old system of simply removing suspect papers isn't particularly transparent.

This is where the revolutionary part comes in. Spearheaded by PubPeer, there is now a site that allows anyone to comment on any publications. This may not seem like much in an age where comment threads at the foot of news articles are commonplace. However, all but a few scientific publishers have steadfastly resisted following suit. PubPeer, which is becoming the Reddit of the science world, is attempting to make post-publication commenting the norm.

It may still be small fry, but bigger fish have taken notice, most notably PubMed, an enormous and well used database of medicine-related articles. When Pubmed, last week, added a commenting facility to its database it gave organised post-publication peer review a huge boost. Given this facility to openly discuss scientific papers, we could be in for a brave new world where scientists are no longer judged on the content of their papers but also on the comment threads that follow. Scientists had better grow thick skins, quickly.

Mark Lorch blogs at www.chemistry-blog.com and www.t2ah.com. You can find him on Twitter @sci_ents

 

Leave a Comment

Required fields are marked *

*

*