Some systematic reviews are confusing. Sometimes this is just poor writing style. Sometimes it\u2019s because the techniques are difficult to grasp (meta-analytic item-response analysis, anyone?) And occasionally it\u2019s because the data don\u2019t seem to add up \u2018right\u2019.<\/p>\n
Take, for example, the systematic review of an imaginary therapy which shows that when given to children with an asthma exacerbation it improves their clinical severity scores yet doesn\u2019t change their peak flow recordings.\u00a0 How\u2019s that work?<\/p>\n
One explanation might<\/em> be \u2018selective outcome reporting bias\u2019. I find it easiest to think of this as a sort of within-study version of publication bias. (To recap, publication bias is the finding that studies with more dramatic outcomes tend to have a greater chance of being published, and be published more quickly, than those showing no or negative effects.) What happens in some instances is that certain favourable outcomes (e.g. clinical severity scores) are detailed in those papers which have \u2018positive\u2019 results, whereas other outcomes (e.g. PEFR) are reported in all studies, regardless of their outcome. \u00a0It can be best assessed by comparing the protocol, detailing which data are to be collected, with the output of the final paper. The description of the potential for such problems is now embedded in Cochrane reviews, but in other studies it is worth examining for when the results don\u2019t all add up.<\/p>\n","protected":false},"excerpt":{"rendered":" Some systematic reviews are confusing. Sometimes this is just poor writing style. Sometimes it\u2019s because the techniques are difficult to grasp (meta-analytic item-response analysis, anyone?) And occasionally it\u2019s because the data don\u2019t seem to add up \u2018right\u2019. […]<\/p>\n