Trish Groves: What does Tamiflu do, and how will we know?

Jonathan Nguyen-Van-Tam, virologist and researcher from the University of Nottingham, told a group of triallists and virologists last week “we must remember why we’re here—because of the controversies. The clinical world doesn’t believe that Tamiflu works. We should assess whether the regulatory approval/product insert for Tamiflu is valid.”

That group, the MUltiparty Group for Advice on Science (MUGAS) was at a workshop in Brussels on 18 June organised by the European Scientific Working group on Influenza (ESWI) and supported by an unrestricted grant from Roche. Led by several of the original Tamiflu regulatory triallists, the workshop heard plenty of evidence to challenge current claims about Tamiflu’s effects. MUGAS decided to plan and conduct individual participant data (IPD) meta-analyses of the randomised trial data—and observational data. That’s quite a remarkable turnaround, given the strength of claims made by some of the same people over the past decade.

BMJ readers will already be very familiar with growing concerns about oseltamivir’s effectiveness. Earlier this year Professor Harlan Krumholz and co-authors concluded in an editorial in the BMJ that “Despite government claims, we should acknowledge the uncertainty surrounding oseltamivir’s effectiveness and the gaps in publicly available evidence. On the basis of the available data, at best the drug shortens symptoms by about a day when used within the first two days of symptoms, but it has no effect on hospital admissions. In addition, trial data from which to draw conclusions about complications and transmission of flu are lacking.”

WHO made particularly firm claims about oseltamivir in August 2009 during the swine flu (H1N1) pandemic. WHO then stated that, “The guidelines represent the consensus reached by an international panel of experts who reviewed all available studies on the safety and effectiveness of these drugs…Evidence reviewed by the panel indicates that oseltamivir, when properly prescribed, can significantly reduce the risk of pneumonia (a leading cause of death for both pandemic and seasonal influenza) and the need for hospitalization.”

That expert panel advising WHO was anonymous: all its members had signed a confidentiality agreement. But at least one member was identified later as Professor Arnold Monto of the University of Michigan, who coauthored several trials of neuraminidase inhibitors back in the 1990s. The same Professor Monto is one of the four convenors of MUGAS, along with Professor Ab Osterhaus (of Erasmus MC University in Rotterdam, a scientific advisor to Roche, another oseltamivir triallist), Professor Menno de Jong (of Academic Medical Center Amsterdam, virologist and researcher) and Professor Rich Whitley (of University of Alabama at Birmingham, professor of pediatric infectious diseases and an adviser on flu to the Obama administration).

Barry Clinch, Principal Clinical Scientist at F. Hoffmann-La Roche, presented at the MUGAS workshop an overview of all studies in the company’s oseltamivir research programme. His slides showed that in all, 18,928 patients had taken part in trials, observational studies of treatment, and studies of prophylaxis; around 11,500 of them treated with oral oseltamivir. Randomised controlled trials (RCTs) had included 4799 adults and 1368 children and, along with some open label studies, a total of 8078 patients had taken part. All but one of the 12 RCTs took place in the late 1990s. Roche’s lab researchers had also done experimental flu studies and clinical pharmacology studies.

Clinch confirmed that investigators were told to always report admissions to hospital during the Roche trials, but warned that “caution should be exercised in interpreting results on hospitalisation.” This is because there were such low event rates: for example in trial WV15671 just 1 of 201 participants was hospitalised. The main intention to treat analyses in the 12 Roche RCTs (for all patients enrolled with flu-like illness and who had at least 1 dose of trial medication) failed to find statistically significant evidence that hospitalisation rates were reduced by oseltamivir when compared against placebo. For the ITTI population (the subset with flu confirmed by culture or >4 fold rise in antibody titre) the overall P value was 0.06 but this was grossly underpowered.

Presenting the key oseltamivir trials that yielded data on flu complications, Clinch explained that a standard case report form was used to collected data on “Secondary illness.” “We didn’t ask physicians to actively look for complications,” he said. “They simply reported them if they thought patients had, for example, sinusitis, otitis media, bronchitis, pneumonia or other chest infections.” Clinicians could also say if patients needed antibiotics or X rays—thereby implying that there might be some secondary illness—but there was no requirement to confirm diagnoses without anything more than a clinical diagnosis. “To be honest, we weren’t that stringent at the time,” acknowledged Clinch. (And, of course, at that time Roche was testing oseltamivir primarily as another antiviral for seasonal flu, not as a lifesaver in a pandemic.) Only two trial protocols required reporting of clinically diagnosed complications (coded as bronchitis, pneumonia, lower respiratory tract complication, or antibiotic prescription occurring >48h after start of treatment with oseltamivir) as a formal secondary endpoint: those among older and at risk participants. Statistical analyses of these outcomes were exploratory, said Clinch.

The ensuing questions and discussion among the MUGAS review board members (some of whom co-authored some of the trials in question) confirmed that “secondary illness” was indeed only a clinical diagnosis, that in most of the trials its reporting was ad hoc, and that Roche does not know the extent of any missing data. Decisions to give antibiotics were made for nonspecified clinical reasons, and, as someone pointed out, antibiotic usage rates will have varied a lot from site to site because of geographical variations in prescribing behaviour.

However, some of the MUGAS virologists said we know from Roche’s original observational studies and from subsequent case series from the H1N1 pandemic that oseltamivir does reduce complications. Do we? Not yet, although Professor Van Tam’s group has conducted a systematic review and IPD meta-analysis of published and unpublished observational study data during the H1N1 pandemic that will be submitted soon for publication. This study was sponsored by Roche but Professor Van Tam said the data agreement gives Roche no access to the data. The authors plan, however, to share the data with other investigators on request.

When asked what proportion of the original Roche oseltamivir research programme had results in the public domain, Clinch said “about 90% in one form or another.” Virologist Fred Hayden, co-author of several Roche trials, asked: “the contentious area is the 8-10 unpublished trials done some time ago. Are there any plans to publish those?” Clinch replied “it’s a good question…we’d like MUGAS to discuss that. Roche has no objection to that.” Professor Osterhaus said that MUGAS wants to analyse the IPD from those trials and to publish the resulting papers in peer reviewed journals.

So, of course, does the Cochrane Acute Respiratory Infections Group, which has been struggling for years to get the unpublished trial data from Roche. That data release has now begun, and the Cochrane group is poring over the partly redacted Clinical Study Reports right now.

So it’s from famine to feast, with two different groups working towards meta-analyses of the unpublished patient-level data on the effects of oseltamivir in flu. Will they come to the same conclusions?