Tracking Guidelines’ Errors
Guest Blog Post
Authors: Primiano Iannone, MD, Monica Minardi, MD, James Doyle, MD
Institution: Emergency Department, Ospedale del Tigullio, Lavagna, Genova, Italy
Email: p.iannone@live.com
Perspective: Wrong guidelines: why and how often they occur
Methods: Wrong guidelines: how to detect them and what to do in the case of flawed recommendations
Despite being used by most physicians to offer the best option of care for their patients, guidelines can often suffer from serious flaws making them untrustworthy, even though considered evidence based tools.
We identified three categories of guidelines’ untrustworthiness: 1) method related, when incorrect methods have been used (including inadequate management of conflict of interests, panel composition ; 2) content related, when there is discrepancy between recommendations and primary evidence which they refer to; and 3) outcome related, in the case of outcomes diverging from those expected by following the recommendation. We considered quality of primary evidence against trustworthiness of guidelines, and identified the need to set a trustworthiness threshold to be reached before adopting a recommendation as true, depending on quality of guideline and the amount of evidence available. Furthermore, we searched the possible causes of guidelines’ untrustworthiness not only amongst the traditional factors commonly considered (conflict of interests, poor methods, panels not representing all of the stakeholders, lack of external and independent assessment of recommendations) but also with regards to the “waste” of biomedical research, as depicted by sir Iain Chalmers, which raises concerns regarding relevance of clinical research and coherence with existing knowledge. Additionally the lack of addressing public health outcomes is considered.
Ultimately, we offered a “safety bundle” to help users to navigate guidelines with confidence, since current quality assessment tools (AGREE, GIN, IOM instruments) and guidelines repositories and databases do not express a quality rating which is directly useful in order to reliably discriminate between right and wrong guidelines.
We identified and collected a substantial number of guidelines untrustworthy for either methods, content, or evidence of unexpected outcomes. We hope the readers will find this approach valuable in highlighting the awareness on flaws and errors, discussing guidelines trustworthiness hence cautiously interpreting their recommendations.