{"id":1198,"date":"2016-11-21T19:50:35","date_gmt":"2016-11-21T19:50:35","guid":{"rendered":"http:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/?p=1198"},"modified":"2017-08-21T09:45:22","modified_gmt":"2017-08-21T09:45:22","slug":"primary-care-with-geoffrey-modest-md-lessons-ive-learned-from-looking-at-the-medical-literature","status":"publish","type":"post","link":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/2016\/11\/21\/primary-care-with-geoffrey-modest-md-lessons-ive-learned-from-looking-at-the-medical-literature\/","title":{"rendered":"Primary Care with Geoffrey Modest MD: Lessons I&#8217;ve Learned From Looking at the Medical Literature"},"content":{"rendered":"<p><strong>By Dr. Geoffrey Modest <\/strong><\/p>\n<p>There have been several concerning issues and\u00a0lessons that I have learned in the process of doing these blogs over the past several years (I am sending out\u00a0this email\/blog as a follow-up to some of the methodological issues and perhaps incorrect assumptions inherent in many clinical studies and their\u00a0application\u00a0to actual patients, as\u00a0noted in the recent blog on placebos. See <a href=\"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/2016\/11\/14\/primary-care-corner-with-geoffrey-modest-md-benefits-of-placebo-for-low-back-pain-and-some-random-thoughts\/\">https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/2016\/11\/14\/primary-care-corner-with-geoffrey-modest-md-benefits-of-placebo-for-low-back-pain-and-some-random-thoughts\/<\/a><\/p>\n<ul>\n<li><strong>Meta-analyses<\/strong>:\n<ul>\n<li>There is\u00a0huge variability in the actual utility\u00a0of meta-analyses in making clinical decisions. these\u00a0analyses\u00a0are mathematical concoctions which try to combine\u00a0different studies with usually very different people (different inclusion\/exclusion criteria, people with different levels\/types\u00a0of comorbidites,\u00a0different ages, different ethnicities, often different doses of the\u00a0med being assessed, even somewhat different outcomes measured). And the meta-analyses themselves have different inclusion criteria (minimum number of people in a\u00a0study that they include, the authors&#8217;\u00a0assessment of the quality of the study). And they use different statistical analyses (e.g. some do propensity score matching as a means to control mathematically for different patient\u00a0baseline characteristics; or\u00a0they may use different basic\u00a0statistical analyses). Also, in some cases the meta-analysis is overwhelmed by a single\u00a0very large study (i.e., a meta-analysis with 10 studies, but the\u00a0one with many more patients will give\u00a0much more statistical weight to that one study, even if the smaller studies were actually methodologically better).\u00a0As a result I have seen almost simultaneous meta-analyses on the same subject\u00a0in different journals coming to different conclusions.<\/li>\n<li>There was a really good article looking at the pyramid of the value of different types of clinical\u00a0evidence\u00a0(see\u00a0<a href=\"http:\/\/ebm.bmj.com\/content\/21\/4\/125.short?rss=1&amp;ssource=mfr\">http:\/\/ebm.bmj.com\/content\/21\/4\/125.short?rss=1&amp;ssource=mfr<\/a> ,\u00a0or\u00a0Evid Based Med2016;<strong>21<\/strong>:125-127\u00a0doi:10.1136\/ebmed-2016-110401\u00a0) which, unlike other &#8220;evidence\u00a0pyramids&#8221; in the literature over the past 20 years, dismissed\u00a0meta-analyses\/systematic reviews, and highlighted, for example,\u00a0that study design itself (i.e. an RCT) does not necessarily mean that it is a &#8220;better&#8221; study and should be the one\u00a0influencing clinical practice\u00a0just because of its design,\u00a0over a good cohort study (they demonstrate this by their schematic pyramid of evidence-based medicine having wavy lines separating the types of studies, instead of straight-line clear-cut separations of the value of studies by their design. and they do not include meta-analyses\/systematic reviews in the pyramid). To me,\u00a0RCTs are clearly limited by their exclusion and inclusion criteria, and suffer from reductionism (see prior blogs, but basically reducing &#8220;n&#8221; patients into some mathematical average of, e.g., a 53\u00a0year-old patient, 35% female, 78% white, 37% diabetic, with no renal failure and 56% on aspirin&#8230;&#8230;&#8221;), and trying to apply the results\u00a0to a totally different individual patient you are treating with different ethnicity, comorbidities, meds, etc.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<ul>\n<li><strong>Guidelines <\/strong>(also not included in the pyramid of the value of evidence-based medicine, above):\n<ul>\n<li>There has been an unfortunate\u00a0evolution of clinical guidelines, with a few dramatic shifts over the decades:\n<ul>\n<li>The older guidelines were written by the NIH or similar governmental organization, with an emphasis on bringing in different experts both within the field and, at least to my experience, some outside of the field (e.g.\u00a0clinical people), and providing a more consistent, less biased,\u00a0and independent\u00a0validation mechanism for the recommendations<\/li>\n<li>Perhaps related to ideological or financial imperatives,\u00a0newer guidelines are more often being channeled back from the governmental agencies\u00a0to professional societies, creating a few problems:\n<ul>\n<li>Guidelines may not reach the same conclusions: e.g. the early versions of the\u00a0Am Diabetic and Am Heart Assn guidelines on blood pressure goal. Then, what is a clinician to do??<\/li>\n<li>The professional societies&#8217; guideline-writing groups\u00a0often do not include practicing clinicians (at least from what I\u2019ve seen), but mostly the higher-ups (i.e., mostly researchers)\u00a0in the professional societies. There is often a significant financial\u00a0conflict-of-interest with many guideline-committee\u00a0members,\u00a0though this is being watched and reported more now than before, more with some professional societies than others. But, beyond those direct financial\/other interests of some of the specialty society leaders,\u00a0I would guess that\u00a0it is not easy\/comfortable for others\u00a0within the societies\u00a0to be\u00a0critical of them (they are the &#8220;leaders&#8221;, with disproportionate influence within the writing committee and within the specialty society)<\/li>\n<li>And there are a huge profusion of guidelines, from all of these societies, to the point that it is pretty impossible to keep up with them<\/li>\n<li>However, I think the real reason that guidelines\u00a0are not considered part of the &#8220;evidence pyramid&#8221; noted above\u00a0is that there is no external validation metric used for\u00a0these guidelines: there are a group of specialists sitting around a table and making recommendations about how we should treat patients, and\u00a0with an inherent conflict-of-interest above and beyond those of\u00a0specific leaders promoting a technique or drug which they may personally benefit from. Is it surprising that the American Urological Association has historically been\u00a0much more aggressive in pushing for PSA screening? Or the American Cancer Society historically pushing for\u00a0more cancer screening? Or the American College of Radiology promoting\u00a0more mammograms?<\/li>\n<li>So, the best\u00a0model to me is reverting to the way guidelines used to be created,\u00a0as currently done\u00a0in other countries\u00a0having\u00a0a single uniform approach to guidelines (e.g. the NICE guidelines in the UK are pretty exemplary to me: very thoroughly researched, with, I think, pretty unbiased and thoughtful\u00a0recommendations), using the best external validation metric to promote the best, least-biased\u00a0recommendations based on known\u00a0data and relatively unbiased expert opinion and informed by practicing clinicians. probably the best we have now in the US\u00a0is USPSTF, though they also have an important-to-know filter of usually needing strong support from RCTs to really\u00a0endorse an approach\u00a0(e.g., see\u00a0<a href=\"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/2016\/08\/30\/1114\/\">https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/2016\/08\/30\/1114\/<\/a> which does not recommend lipid screening in adolescents, despite what I think is pretty compelling though circumstantial evidence, basically\u00a0because there are no good 30-40 year studies following 12 year-olds, randomized to diet\/exercise\/perhaps meds at some point, and looking at\u00a0clinical outcomes).<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<ul>\n<li><strong>Using on-line sources for quick guidance<\/strong> (e.g. Up-To-Date, etc.)\n<ul>\n<li>These are also not on the &#8220;evidence pyramid&#8221;, for reasons similar to the guidelines issue: the entries\u00a0are the non-validated opinions of a few individuals about how to evaluate, diagnose and treat patients. There are no upfront disclosures of commercial interest (if you click on an author&#8217;s name, then on disclosures in Up-To-Date, you can get the info, but it is a few clicks away, and, I would guess that a busy clinician looking for a quick answer probably does not do this a lot. And then the information is that the author gets money from perhaps a specific\u00a0drug company. And, I would also guess, most of us\u00a0primary care clinicians have no idea which meds that drug company makes and therefore which suggested med in the Up-To-Date review\u00a0might be promoted more&#8230;).<\/li>\n<li>That being said, I do not know a clinician (including myself) who does not use one or more of these sources pretty often, to get quick guidance about what to do with the patient in front of them&#8230;. \u00a0it is so easy, typically has a review of the relevant studies, and gives very clear guidance. The only issue is bias and reliability&#8230;..<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<ul>\n<li><strong>Misquoting references<\/strong>\n<ul>\n<li>As mentioned in a few blogs, sometimes the articles misquote\u00a0references, claiming incorrectly\u00a0that a previous study came to a certain conclusion. So, it is useful\u00a0to check the original article when an article\u00a0makes a statement about another article\u00a0that seems out-of-line. This is a lot of extra work, though way easier than it used to be (often you can click on a hyperlink of the reference, or do a quick online search. Easier than going to the library&#8230;)<\/li>\n<li>Even more commonly (still not very common), articles sometime\u00a0make\u00a0reference to a citation which is incorrectly cited\u00a0(i.e., you look at the article cited and it has nothing to do with the author&#8217;s point??An error by the author\/journal editor\u00a0in making sure that the citation matches??)<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<ul>\n<li><strong>Supplemental materials<\/strong>\n<ul>\n<li>Oftentimes, some of the most important material is relegated to\u00a0the supplemental material (including important subgroup analyses,\u00a0methodologic issues, data backing up some of the article&#8217;s conclusions,\u00a0conflicts-of-interest, etc.)\u00a0which really give lots of insight into the real value of an intervention. These are only accessible online (an issue if you do not subscribe to that journal)\u00a0and are, I think, a significant impediment for many clinicians to access. In cases where I cannot get a specific article and have emailed the author for a copy, I only get the PDF, and unless I want to pay $30-50 to get the article through the journal (which I am not), I cannot see the supplementary materials.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<ul>\n<li><strong>Using not-so-relevant clinical endpoints<\/strong>\n<ul>\n<li>There has been a trend to using composite endpoints (perhaps to make the likelihood of an intervention&#8217;s\u00a0benefit higher and more likely to be statistically significant) which just don&#8217;t make sense, such as combining a really important outcome\u00a0with much less important ones. For example, a recent blog looked at CPAP for OSA (see\u00a0<a href=\"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/2016\/10\/03\/primary-care-corner-with-geoffrey-modest-md-cpap-does-not-reduce-cardiovasc-risk\/\">https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/2016\/10\/03\/primary-care-corner-with-geoffrey-modest-md-cpap-does-not-reduce-cardiovasc-risk\/<\/a> ), assessing CPAP utility for\u00a0the composite endpoint\u00a0of hard cardiovascular events plus the development of hypertension. If there were benefit for\u00a0significant hard cardiovascular events, I would be quite inclined to suggest CPAP for my patients. But if\u00a0CPAP\u00a0only\u00a0decreased\u00a0hypertension a little (but statistically significant), I would treat that\u00a0by reinforcing lifestyle changes, or using a med if needed, and would not prescribe\u00a0CPAP. Or, another example:\u00a0the ADVANCE\u00a0study, which looked at tight blood sugar control on the effects on hard CVD outcomes plus diabetic nephropathy. This seems pretty silly. We know from many studies that tight control helps prevent\u00a0diabetic nephropathy. The more important clinical issue\u00a0is cardiovascular benefit or harm. And adding a known quantity of decreasing nephropathy into the &#8220;composite&#8221; endpoint just dilutes\/distorts the results. This study\u00a0really highlights the general\u00a0issue of\u00a0lumping together non-equivalent outcomes (it is hard to argue that developing early nephropathy is somehow equivalent to, and should be numerically added to, CV deaths or nonfatal strokes; or in many other studies, lumping together all-cause mortality with need for an additional clinical\u00a0procedure).\u00a0I raise these\u00a0issues as examples, but this is really a very common finding. And this approach of combining endpoints may be worse now, since a large percent of the studies done are designed by drug companies, etc., which have a vested interest in the most positive outcome.\u00a0And sometimes one cannot disaggregate the individual outcomes without access to the supplementary material&#8230;.<\/li>\n<li>As I have railed about in many blogs, I am really concerned that the FDA accepts surrogate endpoints for some clinical diseases. The most evident one is using A1C as the end-all for new diabetes meds. Personally, I don&#8217;t really care so\u00a0much about the A1c, just what really happens to patients. Many of the new drugs approved do decrease the A1c (though only a little, in most cases), yet have significant and serious adverse reactions (see many\u00a0blogs in\u00a0<a href=\"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/category\/diabetes\/\">https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/category\/diabetes\/<\/a>\u00a0) which undercut their utility significantly (e.g., as cited in many prior blogs: rosiglitazone does well in lowering A1C, just unfortunately increases cardiac events&#8230;)<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>So, I am writing this blog mostly because I have been doing these blogs for several years now, have been reading lots of articles, have the (perhaps)\u00a0benefit of seeing the evolution over decades\u00a0of clinical research\u00a0and the medical-political-social-economic structure of both the research being done and how it is reported, and am pretty frequently\u00a0struck by some of the not-often-acknowledged gaps and concerns of\u00a0that literature and its effect on clinical practice. I would recommend reading the &#8220;evidence pyramid&#8221; article in the BMJ Evidence-Based Medicine journal referenced above, since it does comment a bit on some of these (and did stimulate me\u00a0to write this). But, of course, I should also comment that\u00a0all of the\u00a0above are my observations (i.e., not validated by an independent group), but at least I have no (i.e., zero) conflicts of interest, other than the bias to a real skepticism in reading articles and guidelines, or of\u00a0being an early adopter of new meds\/procedures&#8230;..<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Primary Care with Geoffrey Modest MD: Lessons I&#8217;ve Learned From Looking at the Medical Literature  [&#8230;]<\/p>\n<p><a class=\"btn btn-secondary understrap-read-more-link\" href=\"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/2016\/11\/21\/primary-care-with-geoffrey-modest-md-lessons-ive-learned-from-looking-at-the-medical-literature\/\">Read More&#8230;<\/a><\/p>\n","protected":false},"author":148,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[14283],"tags":[],"class_list":["post-1198","post","type-post","status-publish","format-standard","hentry","category-archive"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/wp-json\/wp\/v2\/posts\/1198","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/wp-json\/wp\/v2\/users\/148"}],"replies":[{"embeddable":true,"href":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/wp-json\/wp\/v2\/comments?post=1198"}],"version-history":[{"count":0,"href":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/wp-json\/wp\/v2\/posts\/1198\/revisions"}],"wp:attachment":[{"href":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/wp-json\/wp\/v2\/media?parent=1198"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/wp-json\/wp\/v2\/categories?post=1198"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/stg-blogs.bmj.com\/bmjebmspotlight\/wp-json\/wp\/v2\/tags?post=1198"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}