Think for a moment about all the scientific articles you’ve peer reviewed throughout your career.
Do you ever find it challenging to make time to perform the review? Do you worry about your reviews being too picky or lenient? Have authors brushed over your comments without really tackling them? Do you ever wonder why you keep agreeing to peer review articles?
With over 2.5 million new scientific papers published each year in over 28 000 scholarly journals, the importance of peer review to the scientific field cannot be understated. The critiques and recommendations—as well as the authors’ responsiveness to them—are the driving forces behind scientific quality control. When done well, peer review is the catalyst that enables journals to accept and disseminate valid and meaningful articles that are destined to have the highest impact for patients.
Despite the importance of peer review, most journals keep it private. As a result, for most published articles, the scientific populace can assess neither the contribution of reviewer critiques and recommendations to the work, nor the quality the authors’ responses. Consequently, we are left to assume that the peer review process was sufficient. That’s a hefty assumption to make.
My study team and I recently experienced a more transparent peer review process through The BMJ with our submission, Age Trends in 30 day Hospital Readmission: A National, Retrospective Analysis. At The BMJ, the authors and reviewers are un-blinded to each other. Moreover, The BMJ’s editors and reviewers comments—along with the authors’ responses to them—are published alongside each accepted manuscript. Experiencing this degree of pre-publication exposure was new to us. We were curious about how it would play out.
The aim of our research was to assess variation across age in the odds of hospital readmission for all people in the US. When reviewing our original article submission, the editors and reviewers made suggestions on how to revise the statistical analysis of the relationship between age and readmission. They suggested that we analyse age in 1 year epochs instead of the larger age groupings (for example, age categories in 5 year increments) that we had originally used. They also suggested that we revise our cohort inclusion criteria to make the study population more generalisable worldwide.
Being responsive to those suggestions meant starting over and re-running all of the analyses. Entering 90 age categories into multivariable analysis on over 31 million patients was computationally intense. Nonetheless, we believed that the suggestions were scientifically sound, justified, and important to answer. So we obliged, much to the chagrin of our cranky, research-computing hardware servers.
The recommendations made by The BMJ’s editors and the peer reviewers led to a major new discovery. The revised analyses showed an abrupt decrease in the adjusted odds of hospital readmission for study participants between ages 64 and 65 years. This is an important finding because, in the US, 65 is the threshold of focus for most federal policies on measurement and financial penalties for hospital readmission. It is also the threshold for one of the most important changes in health insurance coverage for people in the US.
My study team and I are grateful that The BMJ’s editors and reviewers put forward ideas that uncovered new findings in our work. We’re also grateful for the transparency of The BMJ’s pre-publication process because it will let readers see how the discovery was made, including how the study findings changed as a result of tackling the peer review recommendations. We find it fascinating to show the evolution of our work in this way.
So, think about this story the next time you’re working hard to peer review a scientific article. Know that your review is important and that you have an opportunity to make a considerable impact. As transparency for peer review continues to increase, so will acknowledgement of the contributions that arise from your thoughtful efforts.
Jay Berry is a hospitalist and health services researcher with the Complex Care Service in the Division of General Pediatrics at Boston Children’s Hospital and Harvard Medical School.
Disclaimer: The views expressed in this commentary represent the views of the author and not necessarily the host hospital or institution.
Competing interests: I receive funds from the US Agency for Healthcare Research and Quality and the Maternal and Child Health Bureau to conduct independent research.