Just as some people proudly announce that they are alcoholics, I’m proud to tell the world that I am a Cochrane reviewer. There cannot be many BMJ readers who don’t know what that means, but just in case it’s a calling, something akin to joining a monastery. In support of my work, I have studied and learned the methodology, actively participated in the meetings, and done the ritualistic dance that marks me as a True Believer – committed both to research synthesis and more directly to a lifetime of updating the “Effect of social franchising on access to and quality of health services in low income and middle income countries” (1). Actually, it is a little like being married and I really do walk around quoting Sir Iain Chalmers on the need for “all research to begin and end with systematic review.”
The complex issues that are the focus of health systems research should not be an exception to this rule. I specialize in reviews of health systems research topics – so more so than conducting a meta-analysis as to which drug works best from a series of randomized controlled trials, I look at how societies can best organize themselves to produce health in the population. But do those of us who conduct systematic reviews of health systems evidence might need new methods and perhaps a new home? We’ve been worrying about this issue for the last few years and a group met in London last week to think about the future.
Part of our problem is illustrated by my Cochrane review – on a pressing health systems topic – being an empty shell. The team and I did a fine job of defining the intervention, implementing a comprehensive search strategy and diligently screening abstracts, but as is the case for so many health systems questions, the methodological requirements of Cochrane reviews resulted in a review for which no evidence yet existed or at least not the appropriate type of evidence. (It is worth noting that the team and I have since completed a second, non-Cochrane review of social franchising – and evidence does exist but mostly from cross-sectional designed studies but the material is very descriptive. We hope to publish this second review soon in an open access journal.)
In that vein, at the Global Forum for Health Research in Havana in November 2009, the Alliance for Health Policy and Systems Research sponsored a session that posed the question: “Do we need a Cochrane-like collaboration for synthesizing health-system evidence?” Of the four presentations in the session, I had been designated the loaded task and subsequently delivered a presentation on the “Merits and Limitations of Cochrane Reviews for Health Systems Research,” which I have learned first-hand. The challenges and methodologies of doing health systems reviews are as diverse as the health systems questions that can be asked. The crux of the issue is that presently there is no one coordinated system for producing health systems reviews and there is not a standard set of methodologies for approaching various types of synthesis for health systems research. (However, you may find many reviews collated from different sources at www.healthsystemsevidence.org.) The Alliance is attempting to lead global efforts to develop a sustainable solution to this problem.
The London meeting was a follow up to the Cuba meeting, and we addressed the question “Do we need an international collaboration for synthesizing health systems evidence?” It was convened by the Alliance and hosted by DfID. Glitterati of the systematic review world attended so that there were representatives of the EPPI Centre, the Cochrane Library, NICE (National Institute for Health and Clinical Excellence), 3IE, the World Health Organization, Harvard School of Public Health, the Campbell Collaboration, Health Technology Assessment, the London School of Hygiene and Tropical Medicine and more. The meeting started with a discussion of the duel need to synthesize health systems research and to translate the synthesized health systems knowledge and make it available to health sector decision makers globally. Both the demand and supply side issues are important, but it became clear that the first priority of the working group needs to be on the synthesis. Moving into issues, we started with the barriers to health systems research synthesis particularly the lack of standardized methodologies so that although such research synthesis is taking place globally, it may be of poor quality or may not be conducted on issues of importance to the would-be users of the systematic reviews. Next, as health systems research is generally on complex topics, the issue of generalizability and transferability of results must be considered and somehow incorporated into health systems reviews and perhaps encouraged to be included/captured in primary research studies. Further, the discussion around methodology seems to scream for the development of a guidebook or handbook of best methodological approaches for different types of questions perhaps similar to the clear guidance provided in the Cochrane Handbook of Systematic Reviews (2). Also, there is poor coordination or capturing of health systems reviews so that there is duplication of efforts. Better coordination between efforts might better serve the policy makers and development community.
I came away from the meeting still undecided as how best to develop or where to host a collaboration for health systems research synthesis. Should such a collaboration sit within an existing synthesis structure or should something new and different be created? Can it be facilitated through a network structure of relevant existing collaborations, institutions and organizations? However, strong synthesis of health policy questions has the ability to not only guide policy making toward best practices but can also identify gaps in the evidence where more primary research is needed – and health is a multi-trillion dollar industry so the lack of synthesized health systems research can be consider an expensive and important lacking. The details of the discussion will be refined and circulated. The preliminary recommendations of this Working Group on Health Systems Research Synthesis will be presented and further discussed at the First Global Symposium for Health Systems Research in Montreux in November (3). These discussions are open, and for more information on the date and time of the session, please write to me at Tracey@icddrb.org.
1. Koehlmoos T, Gazi R, Hossain S, Zaman K (2009) Effect of social franchising on access to and quality of health services in low- and middle-income countries. Cochrane Database of Systematic Reviews, Issue 1. http://www2.cochrane.org/reviews/en/ab007136.html
2. Cochrane Handbook of Systematic Reviews.
3. First Global Symposium for Health Systems Research.
Tracey Koehlmoos is programme head for health and family planning systems at ICDDR,B and adjunct professor at the James P. Grant School of Public Health, BRAC University.