Recently, the US Food and Drug Administration (FDA) Commissioner Scott Gottlieb, announced new steps that the agency is taking to enhance transparency of clinical trial information. The FDA is launching a pilot programme whereby selected portions of clinical study reports (CSRs) for nine recently approved new drug applications will be posted on the agency’s website. Furthermore, to foster easier tracking of the progress of drug development, the FDA will use the ClinicalTrials.gov identifier number to link trials to FDA communications about specific drugs, including product labelling and the advisory committee meeting materials.
While it is tempting to say that this is “a step in the right direction,” the narrow scope of the pilot is cause for concern. To begin with, the pilot programme is voluntary: the sponsors decide whether to share their CSR information and what to share. It is limited to just nine recently approved products, and not all trials sponsors submitted as part of their New Drug Application, but only the ones deemed “pivotal.” CSRs for unapproved or withdrawn products will remain buried, and CSRs describing trials for unapproved indications of products on the market (i.e., “off label” use) will also remain out of sight. There is no arrangement for sharing CSRs of old so-called “legacy” drugs, i.e. the majority of drugs in use today. CSRs will also be redacted prior to release, which may (or may not) limit their utility for clinicians, researchers, and the public.
A CSR contains an unabridged and comprehensive description of the clinical problem, design, conduct, and results of a clinical trial, following a structure and content guidance prescribed by International Conference on Harmonisation. Because this report also incorporates tables and figures, with appendices containing the protocol, statistical analysis plan, sample case report forms, and patient data listings (including narratives of all serious adverse events), it can be thousands of pages in length. CSRs differ from trial datasets (i.e., electronic individual patient data [IPD]) in that they are paper (or mostly PDF) documents and most data are already aggregated.
To obtain marketing approval of drugs for a specific indication, pharmaceutical companies submit CSRs and other required materials to regulatory authorities. The pilot programme marks the first time the FDA is proactively releasing CSRs. Although one could always request CSRs through the Freedom of Information Act, the FDA has historically avoided release. In stark contrast, the European Medicines Agency has released millions of pages of CSRs since 2010 under its Policy 0043, and even more now under its additional Policy 0070. Up until now, Health Canada, despite its commitment to transparency, has considered CSRs “confidential business information.” Of note, the comparability and consistency of information contained in CSRs for the same drug-indication pairs, and the “pivotal trials” submitted to multiple regulatory authorities is unknown.
For those who believe health care decisions should be based on the totality of evidence, CSRs have become a gem of information for evaluating the efficacy and safety of drugs because they are full of details that are often missing in the public domain. Above all, the protocols and results of many trials, including those submitted to regulatory authorities, are not published. Furthermore, outcomes associated with positive or statistically significant results are more likely to be reported than outcomes associated with negative or null results. When journal articles were compared to CSRs, we (and others) have found that the information in the public domain cannot be trusted at face value. Outcomes are “switched”. Efficacy results are “cherry picked”. Adverse events, including the most serious ones, are grossly under-reported in the publications. Inaccurate and selective reporting puts the health of the public at stake. Releasing CSRs enables independent scrutiny, for example to identify outcomes assessed but not presented to the public, one central step towards protecting and advancing public health.
While I applaud and welcome the initiatives that FDA is taking to enhance the transparency of its decision making, how, I wonder, can the scientific community raise their game in making use of the wealth of information contained in CSRs? Until now, it is debatable whether the value of CSRs is fully appreciated by the average clinicians. While methodological investigations have been carried out by a handful of academic researchers who are keen on fixing the pervasive problem of selective reporting and transforming the landscape of clinical trial data sharing, very few systematic reviews on drug interventions have included data from CSRs. One recent survey of 160 systematic review authors (over 7,000 invited) found that only 20 authors requested regulatory data and only 12 included CSRs in their systematic reviews. This observation implies that systematic reviewers need guidance, training, and resources to disentangle the rich information contained in CSRs. Likewise, evidence users, particularly those who may believe selective reporting is a story of the past, should keep a vigilant eye on claims made in the published literature about the efficacy and safety of drugs.
Before full-scale release of CSRs, a pilot programme is essential to overcome logistical challenges of a future programme and collect user feedback. Can people find what they are looking for? Are the redactions ‘reasonable’ (i.e., are they necessary to protect participant privacy and legitimate commercial interests)? Will the redacted and abridged CSRs include sufficient details that help users judge the rigor and findings from trials (e.g., effectiveness and adverse event outcomes)? How do CSRs released by FDA compare with those released by EMA? Is voluntary submission sufficient to achieve the transparency goal? I urge the scientific community to participate in the FDA’s pilot programme and comment on the ease of access, as well as usability and completeness of data in the released CSRs.
The Blueprint for Transparency recommends 18 ways that FDA can be more transparent, which, when implemented, will lead to “safer and more effective medical products, with lasting benefits for clinical care, scientific progress, and public health”. I hope additional actions from FDA, suggested by the Blueprint, are still to come.
Tianjing Li is an associate professor, Center for Clinical Trials and Evidence Synthesis, Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health.
Competing interests: My ongoing research is funded by a range of federal agencies, including the National Eye Institute, National Institutes of Health (NIH); the Agency for Healthcare Research and Quality; the Patient-Centered Outcomes Research Institute; and the U.S. Food and Drug Administration (FDA). The funding from FDA supports a project that identifies and quantifies patient preference information for a class of glaucoma surgical devices, which is unrelated to the current topic. In the past, I have received research funding from some of the same agencies, as well as the National Library of Medicine, NIH; National Institute of Aging, NIH; National Center for Advancing Translational Sciences, NIH; the Cochrane Collaboration; and the Johns Hopkins Bloomberg School of Public Health. I have not received any funding from any non-public sources.