Humans tend to remember their pleasant and positive experiences and forget those events which were disagreeable or negative. We are not natural talents at coping with cognitive dissonance, and therefore our minds select the memories and evidence that we are most comfortable with. This trait can seriously impede our objectivity. In extreme cases, it can even obscure the truth. This is one reason why we often cannot trust our own experience but need scientific tools to establish the facts.
Whenever there is a sizable set of scientific data, it is likely that not all of them yield entirely the same result. In situations where the evidence is mixed, it is usually best to first define what type of evidence is most likely to be reliable and subsequently consider the totality of such data. This usually means evaluating all randomised clinical trials of a given therapy for a given condition, an approach best realised in systematic reviews or meta-analyses.
I estimate that around 1000 such assessments of alternative medicine (AM) are available today. Those that are rigorous often arrive at disappointingly negative conclusions such as “the evidence from rigorous studies of therapy x for condition y is inconclusive.” But this is hardly the message that enthusiasts of AM want to hear.
When plagued by cognitive dissonance, believers often do not employ science to solve their dilemma but resort to cherry-picking the evidence, i.e. they select those studies which, for whatever reason, generated a (false) positive result. Subsequently, they publish and promote the findings of this process. Unable to scrutinize the procedure, the unsuspecting public as well as journalists, politicians, and some uncritical healthcare professionals might then be impressed by this seemingly sound evidence. Cherry-picking has thus become endemic in AM and is a popular strategy for AM-enthusiasts to mislead us about the evidence.
Some AM-enthusiasts even go one extraordinary step further claiming that those scientists who conduct rigorous systematic reviews cherry-pick their data. Rigorous reviews usually include only certain studies, i.e. those of high quality. Thus, some AM-proponents claim that such reviews are based on highly selected data rather than on all of the available evidence.
If we use science like a drunken man uses a lamp-post – for support rather than enlightenment – the result will be an obfuscation of the truth which can only be detrimental to good healthcare.
Edzard Ernst is a professor of complementary medicine at the Peninsula Medical School, Exeter.