Comments stimulated by: Saramago et al. BMC Med Res Methodol 2016
This week a new finding in the acupuncture field was published in rather unlikely journal. BMC Medical Research Methodology is one of the Biomed Central range of open access online journals, and the paper principally describes a new method within network meta-analysis for analyzing data from continuous variables using individual patient data.[1]
Briefly, meta-analysis allows summary data from different studies (two-way comparisons from randomized controlled trials – RCTs) to be combined in order to reduce statistical uncertainty, and assess other aspects of the data such as heterogeneity or the likelihood of publication bias. Combining summary data has limitations because the original trials may not report data in the same way, and may not use the most powerful statistical analysis in the first place. Hence there is value in using the raw (individual patient) data and reanalyzing with the same statistical method for each trial before combining the results (pooling).[2]
Network meta-analysis is a method for combining data from multiple two-way comparisons of interventions so that both direct and indirect comparisons between interventions can be performed. For indirect comparisons there must be a common node (or intervention). In CG150 a limited network meta-analysis used placebo as a common node to compare acupuncture directly with topiramate, and concluded that the latter was twice as good as the former.[3] For this to be a valid analysis, sham acupuncture would have had to be the same as placebo topiramate, yet the absolute data seemed to indicate that sham acupuncture was associated with a higher responder rate than the real drug itself, let alone the placebo version.[4] Subsequent analysis of shams in migraine clearly indicate that sham acupuncture and sham surgery significantly outperform all other shams and placebos.[5]
Enough of the sour grapes over the NICE view that topiramate is twice as good as acupuncture, and back to the new paper… This paper by Saramago et al used the data from the Acupuncture Trialists Collaboration that was reported in the first individual patient data meta-analysis (IDPM) in the field by Vickers et al.[2] But unlike the two-way comparisons of this first IPDM, which separately compared acupuncture with sham and acupuncture with no acupuncture controls, Saramago et al were able to simultaneously include all comparisons in a single network. This results in statistically robust data that for the first time compares sham acupuncture with usual care (or ‘no acupuncture’ control) as well as giving a higher degree of reliability to the main comparisons of acupuncture with sham and acupuncture with usual care.
So this brings us to the unexpected new insight that is alluded to in the title of this blog, and merely an incidental finding of the analysis. The results of the network meta-analysis demonstrate that acupuncture is superior to usual care for pain and health related quality of life (HRQoL) – the latter being measured with the EQ5D or converted from other measures such as the SF36, SF12, VAS pain, WOMAC etc. Acupuncture is also superior to sham acupuncture for pain (apart from the headache subset), but not unequivocally superior in terms of HRQoL (95% confidence intervals cross zero). The rather fascinating result is that sham acupuncture is unequivocally superior to usual care in terms of HRQoL, but not consistently superior in terms of pain – see Figure 2 from the paper below.
A rough and ready summary would be: acupuncture is superior to usual care in both outcomes; the difference between acupuncture and sham acupuncture appears greater in terms of pain than HRQoL; and the difference between sham acupuncture and usual care appears greater in terms of HRQoL than pain… Fascinating! This will give mechanistic advocates like me plenty to postulate over the differential effects of gentle versus standard acupuncture approaches in different circumstances.
Postscript note 1:
Usual care is a term used in the paper by Saramago et al, and it refers to the same data set labeled by Vickers et al as no acupuncture controls. This group includes a variety of interventions that could be described as standard or conventional care. Some involved quite intense treatment regimes, and others could be seen as more of a background usual care that might be common to all groups (ie including acupuncture and sham acupuncture groups).
Postscript note 2:
Sham acupuncture mostly involves needling superficial tissues and has similarities with gentle forms of acupuncture. Non-penetrating sham or ‘placebo’ needles are blunt ended, and often cause significant discomfort and can penetrate the skin. It seems clear that sham acupuncture is not synonymous with the term ‘placebo’.
References
1. Saramago P, Woods B, Weatherly H, et al. Methods for network meta-analysis of continuous outcomes using individual patient data: a case study in acupuncture for chronic pain. BMC Med Res Methodol 2016;16:131. doi:10.1186/s12874-016-0224-1
2. Vickers AJ, Cronin AM, Maschino AC, et al. Acupuncture for chronic pain: individual patient data meta-analysis. Arch Intern Med 2012;172:1444–53. doi:10.1001/archinternmed.2012.3654
3. NICE guideline on headaches: diagnosis and management of headaches in young people and adults. 2012.http://guidance.nice.org.uk/CG150
4. White A, Cummings M. Inconsistent placebo effects in NICE’s network analysis. Acupunct Med 2012;30:364–5. doi:10.1136/acupmed-2012-010262
5. Meissner K, Fässler M, Rücker G, et al. Differential effectiveness of placebo treatments: a systematic review of migraine prophylaxis. JAMA Intern Med 2013;173:1941–51. doi:10.1001/jamainternmed.2013.10391
Declaration of interests
I am the salaried medical director of the British Medical Acupuncture Society (BMAS), a membership organisation and charity established to stimulate and promote the use and scientific understanding of acupuncture as part of the practice of medicine for the public benefit.
I am an associate editor for Acupuncture in Medicine.
I have a very modest private income from lecturing outside the UK, royalties from textbooks and a partnership teaching veterinary surgeons in Western veterinary acupuncture. I have no private income from clinical practice in acupuncture. My income is not directly affected by whether or not I recommend the intervention to patients or colleagues, or by whether or not it is recommended in national guidelines.
I have not chaired any NICE guideline development group with undeclared private income directly associated with the interventions under discussion. I have participated in a NICE GDG as an expert advisor discussing acupuncture.
I have used Western medical acupuncture in clinical practice following a chance observation as a medical officer in the Royal Air Force in 1989. My opinions are formed by data that spans the range of quality and reliability, much of which is in the public domain.
I have a logical mistrust of the motives of anyone who advertises an interest or hobby in being a ‘Skeptic’, as opposed to using appropriate scepticism within their primary profession, or indeed organisations that claim to promote generic ‘science’ as opposed to actually engaging in it.