{"id":1039,"date":"2016-05-02T17:40:14","date_gmt":"2016-05-02T17:40:14","guid":{"rendered":"http:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/?p=1039"},"modified":"2017-08-21T10:55:26","modified_gmt":"2017-08-21T10:55:26","slug":"primary-care-corner-with-geoffrey-modest-md-evidence-based-medicine-what-are-its-limitations","status":"publish","type":"post","link":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/2016\/05\/02\/primary-care-corner-with-geoffrey-modest-md-evidence-based-medicine-what-are-its-limitations\/","title":{"rendered":"Primary Care Corner with Geoffrey Modest MD: Evidence Based Medicine &#8212; What are its limitations?"},"content":{"rendered":"<p><strong>By Dr. Geoffrey Modest <\/strong><\/p>\n<p>The journal Evidence-Based Medicine (that&#8217;s the one that posts my blogs, part of BMJ)\u00a0just came out with an interesting article challenging the biases inherent in evidence based medicine (EBM) which ultimately can distort the conclusions (see Seshia SS, et al.\u00a0Evid Based Med 2016; 21: 41). They reference a\u00a02014\u00a0BMJ analysis of EBM, noting\u00a0its pluses and minuses (see\u00a0Greenhalgh, T.\u00a0BMJ 2014;348:g3725). The pluses are that EBM has been around for 20 years, has led to the development of more evidence-based reviews such as Cochrane Collaborations, as well as a slew of guidelines more based on specific methodological\u00a0scientific criteria, and in many ways has elevated the basis for conducting more rigorous studies. But there are several minuses which are important to understand in order to interpret the results. Per<strong> the 2014 article<\/strong> (with some of my comments embedded):<\/p>\n<ul>\n<li>Distortion of the evidence based brand: by this they mean that drug\u00a0and medical\u00a0companies have played such a pivotal role in designing research studies\u00a0that they are able to push using surrogate markers as the important outcome\u00a0(e.g. A1C in diabetics), define\u00a0inclusion and exclusion criteria to best show efficacy (though these may really undercut the applicability of the results to regular old patients), and selectively publish positive studies<\/li>\n<li>Too much evidence\/too many guidelines:\u00a0citing a study from 2005 (when there were many fewer guidelines than now), in a\u00a024 hour period\u00a0they admitted 18 patients with 44 diagnoses;\u00a0to read the\u00a0national guidelines on these diagnoses included 3679 pages and an estimated reading-time of\u00a0122 hours. And, I would add further that these guidelines may well be inconsistent with each other (e.g. different blood pressure goals in the American Diabetes Assn vs JNC8 guidelines).<\/li>\n<li>Marginal gains: most of the major therapies have been found (low-hanging fruits), e.g. HIV drugs, H Pylori treatment, statins. Newer trials are often overpowered, allowing them to find statistically significant findings which are not very clinically significant. And, I would add:\u00a0these studies are often pretty short or stopped early, showing small absolute benefit but too short to pick up longer term harms of therapy<\/li>\n<li>Overemphasis on following algorithmic approaches: by overemphasizing specific targets (e.g. A1C in diabetics), clinicians may not pay enough attention to the really important patient issues (the depression, domestic violence, important\u00a0social\u00a0or other medical\u00a0issuesin the patients\u2019 lives). And incentivizing these mechanical issues (ordering A1C&#8217;s) or dealing with pop-ups or care prompts in electronic medical records, may undercut our ability or time spent\u00a0to really help the main problems of patients<\/li>\n<li>Poor fit in those with\u00a0multimorbidity: many of these EBM studies were done in patients with predominantly\u00a0one condition (e.g., by excluding those with renal failure, cancer, etc.). Taking care of patients with multiple ongoing diseases leads to several issues not addressed in the studies: e.g.,\u00a0drug interactions, or polypharmacy (especially an issue as\u00a0our patient populations are getting older and getting more chronic diseases)<\/li>\n<\/ul>\n<ul>\n<li><strong>The 2016 EBM journal article<\/strong> expands this and\u00a0develops more of a framework to understand the cognitive biases in the medical literature, noting that there may well be combinations of biases in any article. They group biases\u00a0as follows:\n<ul>\n<li>Conflicts of interest:\n<ul>\n<li>Financial, nonfinancial (e.g. desire for promotion, prestige), and intellectual (driven by strong personal belief that\u00a0could distort the study)<\/li>\n<\/ul>\n<\/li>\n<li>Individual or group cognitive biases:\n<ul>\n<li>Self-serving bias (affected by group\/organizational motives), confirmation bias (favoring evidence that supports one&#8217;s\u00a0preconceptions), in-group conformity (increased confidence in a decision if in agreement with others, similar to groupthink, where opposing views are discouraged), reductionism (reducing complex or uncertain scenarios into simple ideas and concepts;\u00a0see further comments below), automation bias (uncritical use of statistical software, decision support systems)<\/li>\n<\/ul>\n<\/li>\n<li>Group or organizational cognitive biases: scientific inbreeding (being trained in the same school of thought or by the same experts), herd effect (unquestioned acceptance of experts; reinforced by social media)<\/li>\n<li>Fallacies\/logical errors in reasoning: planning fallacy (incorrectly estimating benefits or costs\/consequences), sunk cost fallacy (inability to change course of study despite problems,\u00a0after\u00a0so much has been invested)<\/li>\n<li>Ethical violations: ranging from subtle statistical manipulations, selective publication, outright fraud\/fabricatrion. There is typically an associated rationalization and self-deception.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>So, a few issues:<\/p>\n<ul>\n<li>These\u00a0articles\u00a0do\u00a0bring up many of the concerns about EBM, despite the rather large positive of its push to make both the literature and its interpretation more rigorous.\u00a0Most of the negatives are\u00a0about inherent biases in designing and conducting studies but also in about being able to apply the results to the individual patient in front of you.<\/li>\n<li>One additional point is that, as the rising tide lifting all the boats, EBM-based guidelines also elevate \u201cexpert opinion\u201d. By this I mean that since we do not have rigorous studies looking at most of the things we do in primary care (or, clinical medicine, for that matter), the guidelines have a lot of expert opinion. It is certainly true that there is a very clear and repeatedly articulated grading system in the reviews\/guidelines\u00a0reflecting the quality of the studies, but often the take-home message is muddled, combining more definitive\u00a0and not-so-definitive\u00a0conclusions all together (i.e., many of the subtleties are lost. We remember the target points highlighted in their conclusions or a\u00a0take-away-message box, which are typically\u00a0of highly\u00a0varying quality). And, to make matters worse, a large % of the \u201cexperts\u201d are under the drug\/medical supply company wings, much more so in the past 20 years of EBM, so there is increased concern about their \u201cexpertise\u201d.<\/li>\n<li>One interesting sideline here is the general approach of medical studies vs anthropologic studies (this comes from a long-lost article I read in the 1980s), which noted that medical studies were fundamentally reductionist: looking at lots of people and averaging their individual characteristics, so that, for example in the ASCOT-LLA lipid study (Sever P, Lancet 2003; 361: 1149),\u00a0a 63.1 yo person, 94.6% white, 18.9% female, having a 24.5% incidence of diabetes,\u00a0blood pressure of 164.2\/95.0, with a median LDL of 212.7, but excluding those with &#8220;clinically important hematological or biochemical abnormalities&#8221;,\u00a0has a 36.0% lower relative risk of developing heart disease after 3.3 years\u00a0on atorvastatin 10mg (and, of course, we will never see that person, and it is in fact a long and tortuous ideological and practical\u00a0leap to apply these results to the individual in front of us)\u00a0vs the anthropologic approach of\u00a0studying a few families intensively\u00a0over 1-2 years and, by really getting to know and understand them,\u00a0to\u00a0generalize these findings to develop\u00a0larger conclusions about culture.\u00a0If you ever get a chance to read some of the really old journals from the bowels of large medical school libraries, many of these medical articles were much closer to the anthropologic approach (detailed case studies of a few patients with a particular clinical presentation). Clearly there are advantages and disadvantages of both of these approaches in terms of understanding disease and treatments, especially given our rather limited understanding of the complexity of the human biological\/psychosocial systems and their interactions, though EBM aggressively promotes the &#8220;reductionist&#8221; method. [The clinical\u00a0case presentations in several of the major medical journals\u00a0does promote the concept of applying what we have learned in the big studies to individual patients. perhaps this approach should be fostered more, though with experienced clinicians with zero ties to drug companies, etc&#8230;.]<\/li>\n<li>But the concept here is: one should be critical of the medical literature, looking carefully at the study design, inclusion\/exclusion criteria, funding sources, and, to the extent we can, assess the likelihood of these underlying biases in distorting the conclusions.<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Primary Care Corner with Geoffrey Modest MD: Evidence Based Medicine &#8212; What are its limitations?  [&#8230;]<\/p>\n<p><a class=\"btn btn-secondary understrap-read-more-link\" href=\"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/2016\/05\/02\/primary-care-corner-with-geoffrey-modest-md-evidence-based-medicine-what-are-its-limitations\/\">Read More&#8230;<\/a><\/p>\n","protected":false},"author":148,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[14283],"tags":[],"class_list":["post-1039","post","type-post","status-publish","format-standard","hentry","category-archive"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/wp-json\/wp\/v2\/posts\/1039","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/wp-json\/wp\/v2\/users\/148"}],"replies":[{"embeddable":true,"href":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/wp-json\/wp\/v2\/comments?post=1039"}],"version-history":[{"count":0,"href":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/wp-json\/wp\/v2\/posts\/1039\/revisions"}],"wp:attachment":[{"href":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/wp-json\/wp\/v2\/media?parent=1039"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/wp-json\/wp\/v2\/categories?post=1039"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/wp-json\/wp\/v2\/tags?post=1039"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}