Four years ago, the US National Institutes of Health (NIH) launched a pilot project to “leverage the social power of the internet to encourage constructive criticism and high quality discussions.” The service allowed readers to post signed comments below any of the 28 million citations indexed in PubMed and was democratically dubbed “Commons”. But earlier this month, the NIH announced it was going to discontinue PubMed Commons.
We are saddened by the news. PubMed Commons offered an avenue for so-called “post-publication peer review,” in which the wider community can openly discuss and critique papers after they are published, not limited by the bounds of traditional letters to the editor. Considering that few journals offer this kind of golden opportunity to interact (The BMJ’s rapid responses being one of the exceptions) PubMed Commons’ platform for open, frank discussion—the very essence of real science—seemed a slam dunk initiative. It was an easy way of commenting in a timely fashion on something of interest or even a way of challenging the written word.
But PubMed Commons is being shut down.
The reasons NIH cites for the closure are disturbing. Despite having millions of users every day and over 28 million citations, only 6,000 articles were commented on. This “low level of participation does not warrant continued investment in the project,” the NIH said, and is a mere fraction of the 54,000 or so comments available on PubPeer, a research discussion platform launched in 2012.
In an interview, Jerry Sheehan, deputy director of the National Library of Medicine (NLM), which runs PubMed Commons, added that “budgets are always tight,” and that “significant effort and cost” would be associated with bringing PubMed Commons along in a planned “new, reengineered PubMed.” Finally, according to Sheehan, NLM is planning to focus its resources by investing in increasing data access and transparency through ClinicalTrials.gov (the trial register run by NLM).
We are curious to understand why PubMed is a place to find studies and other platforms are the place to criticise them. PubPeer told Retraction Watch that one of the reasons for the low level of commenting on PubMed Commons may be that since 2013 PubPeer allows anonymous posting of comments while the Commons did not. According to the PubPeer website, this “led to the exposure of unexpected levels of research misconduct”.
We think it is a shame that Commons will be no more, although the comments will remain accessible (both through PubPeer, which mirrored the Commons content, and also via download from the NIH’s website). We hope PubPeer, a California registered not-for-profit initiative stays afloat, as we are contemplating pursuing post-publication peer review as a possible radar screen for distorted trial publications. Here’s the plan: soon we will be launching our RIAT Support Center. RIAT stands for Restoring Invisible and Abandoned Trials, and is an initiative we first outlined in 2013. The basic idea is to identify clinical trials which have never been published (invisible), or trial publication that report trials in a biased or distorted manner, and for which no one takes responsibility to correct the record (abandoned). Once the trial is identified, following a set of transparent procedures, the trial is reported in a way that allows a full assessment of the data, i.e. it is restored. Four restorations have already been published (CEA Second-Look Trial, Study 329, the 8-Way Trial, and Diclectin pivotal trial) and we will have funds to support more over the next three years.
Identification of worthy targets is therefore the start and we plan to use different means, including public comments, to do so. The types of discussions and contributions which are available in Commons-type sites will alert us to a possible need to correct the written word. This is a worthy goal which becomes even more worthy as every day goes past, with another misconduct story or discussion about conflicting interpretations of the same underlying data.
So rest in peace PubMed Commons, and thank you.
Tom Jefferson, honorary research fellow, Centre for Evidence Based Medicine, Oxford.
Peter Doshi, associate editor, The BMJ.
Competing interests: PD and TJ are funded by the Laura and John Arnold Foundation to run the RIAT Support Center (2017-2020) discussed in this article. See http://www.bmj.com/about-bmj/editorial-staff/peter-doshi for a more compete list of disclosures.