After attending a webinar on the European Medicines Agency’s (EMA) new 0070 policy, which formalises the release of regulatory data held by the EMA, some of my earlier doubts have been addressed by what seems to be a general EMA commitment to openness (see my first blog on this webinar). For example, following a remark and a line in a webinar slide, I repeatedly questioned the EMA about the existence of “two data sets.” But I was told that the data sets (the one for scientific review by the EMA and the one to be published on the EMA website) are “mirrors” differing only in the redactions. The similarity will have to be certified by each manufacturer when submitting both versions. The EMA will not be carrying out a manual check of the content.
The radical nature of the EMA’s policy lies in its offer of routine access to full clinical study reports (CSRs) with (hopefully) minimal redactions just a few days after drug approval (or rejection). CSRs offer us the possibility of delving into and cross checking all aspects of trials: from protocol to informed consent forms, from intended analysis methods to those which were actually used, to templates for participant data collection. Some other aspects of the policy and its implementation are revolutionary, such as access to CSRs of drugs that did not make it to the European market.
The thousands of pages provide a hitherto rarely tapped bonanza of detail and the possibility to address reporting bias—the biggest threat to contemporary, ethical research synthesis.
This brings me to what set off this modern revolution: the stubbornness of the Nordic Cochrane Centre not to bow to industrial secrecy culture and the European Ombudsman’s support for this stance. The trigger of what really is beginning to look like a revolution is, in fact, research synthesis.
Accordingly, I would have expected my research community to follow developments closely and to be eagerly awaiting the release of the documents. In addition, I would have expected research synthesisers to have been accessing CSRs since the start of the retrospective policy (January 2010) and to be busy modifying practices in view of the huge increase in details and workload brought by the release of full CSRs and attendant documents. I can find very little evidence of this.
In one unofficial audit, the runaway beneficiary of the new policies in numerical terms is pharma itself, followed by the “media”—an ill defined term probably including communication companies working for pharma and “legal.” Academics and researchers came a poor fourth in the number of requests and pages released. In a follow-up audit, 17 of the 33 requests in the academic/research institute category in the first quarter of 2013 were lodged by the same person.
In practical terms, our Cochrane review on neuraminidase inhibitors for influenza appears still to be the only Cochrane review based on a complete regulatory trial programme.
Another important aspect is the credibility of the whole evidence based medicine movement, which to this day is still relying on journal articles for its evidence “building blocks.” These have been shown by steadily increasing evidence to be biased and incomplete. In a way, this is not really surprising if you are trying to cram thousands of pages into the average length publication. It is clear that this credibility will continue to be completely undermined if my colleagues do not switch their evidence paradigm from journals to regulation. The added length of time required to do a CSR based systematic review, fits well with a “fewer, better reviews” philosophy, which is seen by some as the next phase of evolution.
The EMA revolution has implications for publishers, journals, and other regulators. I have addressed some of those here. Will any researcher really read a 10 page account of a trial behind a paywall when the full CSR and attendant documents are a click away?
Regulators will also be in a difficult position, with the Food and Drug Administration (FDA) left looking like a stranded whale wrapped in a sort of pre-New Deal commercial secrecy isolationism. The FDA does not release CSRs and the public can only access FDA reviews of the market authorisation application (which it calls NDAs or new drug applications). These, like all reviews, do not tell you everything about all the trials in the programme, but represent the FDA’s take on the one or two pivotal trials that regulators looked at in depth with sponsors’ agreement. We tried to reconstruct a CSR narrative with sufficient detail to allow full appraisal from FDA medical officers reviews (MORs) in our neuraminidase inhibitors review. We failed completely because of a lack of sufficient information in the MOR and other documents available from Drugs@FDA. Use of FDA MORs instead of full CSRs risks swapping one incomplete source (journal articles) for another (FDA reviewers’ comments).
Lastly, if pharma provide full and detailed study reports, why should other sponsors not have to do so, especially big private funders and the medical devices industry? This obvious oversight should be the next target for campaigners.
So, the answer to my title question is: “It does not look like it.”
The first blog on this webinar: The EMA revolution gathers pace
Tom Jefferson, reviewer, Cochrane Acute Respiratory Infections Group, 00187, Roma, Italy.
Competing interests: TJ was a co-recipient of a UK National Institute for Health Research grant (HTA – 10/80/01 Update and amalgamation of two Cochrane Reviews: neuraminidase inhibitors for preventing and treating influenza in healthy adults and children—http://www.nets.nihr.ac.uk/projects/hta/108001).
TJ receives royalties from his books published by Blackwells and Il Pensiero Scientifico Editore, Rome. TJ is occasionally interviewed by market research companies for anonymous interviews about Phase 1 or 2 pharmaceutical products. In 2011-2013, TJ acted as an expert witness in a litigation case related to oseltamivir phosphate; Tamiflu [Roche] and in a labour case on influenza vaccines in healthcare workers in Canada. In 1997-99 TJ acted as a consultant for Roche, in 2001-2 for GSK, and in 2003 for Sanofi-Synthelabo for pleconaril (an anti-rhinoviral, which did not get approval from the Food and Drug Administration). TJ was a consultant for IMS Health in 2013, and in 2014 was retained as a scientific adviser to a legal team acting on the drug Tamiflu (oseltamivir, Roche). In 2014-15 TJ was a member of two advisory boards for Boerhinger and is in receipt of a Cochrane Methods Innovations Fund grant to develop guidance on the use of regulatory data in Cochrane reviews. TJ has a potential financial conflict of interest in the investigation of the drug oseltamivir.