The publication of a summary of the benefits and harms of breast cancer screening in the Lancet, with the full report to follow, raises the question of whether referring this to NICE would have made any difference. The most obvious difference is that NICE would have gone on to appraise cost effectiveness. The final sentence of the review states: “ The panel also believes that the overall cost effectiveness of the UK breast cancer screening programmes needs to be reassessed in view of this report.”
In assessing the clinical effectiveness of technologies NICE requires a systematic review of the evidence. The review panel, assembled by the Department of Health and the charity Cancer Research UK, reviewed the extensive published work which included 3 systematic reviews: by the Cochrane Collaboration, the US Task Force, and the Canadian Task Force. Since all of these used the same set of 11 randomised trials (sometimes reduced to 8 as several were linked), this can be taken as a systematic review. A systematic review would have led to the same trials.
The 2012 review agreed with the previous estimates of screening leading to a reduction in breast cancer mortality of around 20%. This was on the basis of excluding one trial. Previous systematic reviews have differed slightly in how many trials they exclude, but came to very similar results. This review helpfully compared its estimate with the other reviews showing substantial agreement. The Cochrane Review’s estimate based on all trials was 19% reduction in breast cancer mortality, but that reduced to 15% when account was taken of the quality of randomisation. An important difference.
On the key issue of overtreatment, the review noted first that no study met the requirement of comparing two matched cohorts of screened and unscreened women followed up to death…”therefore estimates of overdiagnosis need some indirect inference from available studies.”
The panel noted that follow-up should continue after the screening period to allow catch-up of diagnoses in the unscreened group. “In principle the extended period of follow up should correspond with the lead time, but the average lead time is debated and is not the same for all cancers.” The panel considered that the definition of an overdiagnosed case and thus the numerator in a ratio, is clear, the choice of denominator has been a source of further variability in reported estimates” with no less than seven approaches. (The ratio referred to is the relative risk of overdiagnois or the percentage overdiagnosed.)
The panel identified three of the eleven trials in which women in the control group were not offered screening at the end of the active period of the trial. Estimates of overdiagnosis were made using these three trials using four different methods. Of the four methods, the panel considered the two most useful to be a population and an individual perspective. The population perspective took “the proportion of all cancers ever diagnosed in women who are invited to screening that are overdiagnosed.” The individual perspective was that for a women invited to screening “the probability that a cancer diagnosed during the screening period represents overdignosis.” Meta-analysis of the three relevant trials (Canada 1, Canada 2, Malmo) put the frequency of overdiagnosis at 11% from a population perspective and 19% from the perspective of a woman invited for screening. The panel noted that these estimates were subject to uncertainty due to being based on trials, and not tailored to the UK screening programme.
How does this compare with the other major reviews? Both the US and Canadian reviews avoided providing an estimate of overdiagnosis, but it was a key feature of the Cochrane Review which put the proportion of individual overdiagnosis at 31%, well above 19% in the review.
This difference in overdiagnosis was not due to different trials as the Cochrane Review used the same three (Canada 1, Canada 2, Malmo). The difference appears to be that the Cochrane estimate of 30% was based on the excess in mastectomies and lumpectomies in the screened group, while the 2012 review was based on “the excess of cancers diagnosed.” When the Cochrane review considered mastectomies alone, it put overdiagnosis at 20%, very close to the review’s estimate. The publication of the full report of the 2012 review should enable this to be checked.
I conclude that first, the reliance of the review on randomised controlled trials is important and welcome. Despite their inadequacies, they show levels of reduction in breast cancer mortality that are widely agreed and less than anticipated by some, notably the Forrest Report which expected a 30% reduction.
Second, the review underlines the importance of overdiagnosis, the extent of which was omitted from all major official reports, including Forrest, US Task Force, and the Canadian review. In fairness, the Forrest report noted that the Two Counties trial found possible overdiagnosis of 20% and stated that “Further follow up is required to find out whether this excess persisted.” The 2012 review concurs with the Cochrane Review in quantifying overdiagnosis. It helpfully distinguishes population and individual perspectives. Its estimate of the individual risk of overdiagnosis, while lower than that in Cochrane (19% v 30%) is very close to the Cochrane estimate of 20% increased risk for mastectomy, the most serious consequence of overdiagnosis.
Third, NICE would probably have taken a similar approach to the review. It would have relied heavily on the randomised controlled trials. It might well have distinguished the individual and population perspectives in relation to overdiagnosis. It might have considered the non trial data studies, but generally NICE relies on trials. As noted above, it would have gone further to consider cost effectiveness, probably from a population perspective.
Finally, the limits of medical research are highlighted. While the population perspective should arguably be preferred for public policy decisions, the assumptions required to take a lifetime perspective are considerable.
See also:
Potential Conflicts of Interest
I have published QALY estimates of the effect of breast cancer screening based largely on the Cochrane Review which could have predisposed me to arguing that the 2012 review is compatible with the Cochrane Review.
James Raftery is a health economist with several decades experience of the NHS. He is Professor of Health Technology Assessment at Southampton University. A keen “NICE-watcher,” he has provided economic input to technical assessment reports for NICE but has never been a member of any of its committees. The opinions expressed here are his personal views.