Deep brain stimulation for Parkinson’s disease is a remarkable therapy. Over lunch a colleague recently described how it transformed her life: from slow shaky dysfunctional movement to almost normal. It is one of those small cadres of treatments that does not need randomized trial evidence to know it works: turn off the stimulator and the Parkinsonian tremor returns; turn it back on and they subside. It “obviously” works, but what makes that obvious?
A common misinterpretation of evidence based medicine is that randomized trials are always needed for all treatments. Some critics of EBM have even suggested randomizing EBMers to be thrown out of a plane with or without a parachute [1]. Partly to avoid forcible enrolment in such a trial, some colleagues and I gathered and analysed examples where we felt that an RCT was unnecessary, and why [2]. But even in early “hierarchies of evidence” Dave Sackett had included at the top “all or none” effects alongside RCTs.
So what makes the parachute effect obvious? First, we know the natural history of falling several thousand feet without one—most folk will die. There are some rare exceptions, such as Alan Magee who fell 22,000 feet and survived as his fall was slowed by crashing through the glass roof of the St. Nazaire railroad station. But this is very rare. Second, we know the chance of survival with a parachute is pretty good. We could even guesstimate the relative odds of survival with and without a parachute. Let’s say 1 in 10,000 folk survive (there have been a handful). Then even for an imperfect parachute which saved 9 of 10, the odds are improved dramatically: (9/10)/(1/10,000) = 9,000 times more likely to survive. That is an effect size well beyond the conceivable bias of confounding (people more likely to be able to survive falls are more likely to wear parachutes). Indeed, we suggested that a relative odds of 10 or more was probably sufficient, so the parachute is well clear of that margin!
Though randomization is not always necessary, that does not mean abandoning all principles of evidence evaluation. We can still use a systematic approach to the evidence. For example, we recently did a systematic review [3] of case studies and case series for the “mother kiss” method of removing foreign bodies from a child’s nose: the mother occludes the other nostril and blows in the child’s mouth, forcing air to push the foreign body out. We found the case studies were all positive (no one reported their single failure), but in the case series the success rate was a bit better than half. So, just as the hierarchy of evidence might suggest, case series rates above the single anecdote. If you have other suggestions for our list, please let me know.
Paul Glasziou is professor of evidence based medicine at Bond University and a part time general practitioner.