{"id":926,"date":"2014-05-14T20:33:00","date_gmt":"2014-05-14T19:33:00","guid":{"rendered":"http:\/\/stg-blogs.bmj.com\/adc\/?p=926"},"modified":"2014-05-14T16:02:52","modified_gmt":"2014-05-14T15:02:52","slug":"underpowered-and-over-here","status":"publish","type":"post","link":"https:\/\/stg-blogs.bmj.com\/adc\/2014\/05\/14\/underpowered-and-over-here\/","title":{"rendered":"Underpowered and over here."},"content":{"rendered":"<figure style=\"width: 282px\" class=\"wp-caption alignleft\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/pbs.twimg.com\/media\/BnLHzXxIYAAqzr6.jpg\" alt=\"\" width=\"282\" height=\"143\" \/><figcaption class=\"wp-caption-text\">From @aLittleMedic<\/figcaption><\/figure>\n<p>We&#8217;re great fans in the Archimedes blog of trying to get people to think about the meanings and impacts of research, like asking <a title=\"What would Jack want?\" href=\"https:\/\/stg-blogs.bmj.com\/adc\/2010\/03\/10\/what-would-jack-want\/\">What would Jack want <\/a>and <a title=\"StatsMiniBlog: Type I and II errors\" href=\"https:\/\/stg-blogs.bmj.com\/adc\/2013\/09\/12\/statsminiblog-type-i-and-ii-errors\/\">not believing p-values. <\/a>One key idea is that of an &#8216;<a title=\"It\u2019s all the same\" href=\"https:\/\/stg-blogs.bmj.com\/adc\/2012\/01\/23\/its-all-the-same\/\">important clinical difference<\/a>&#8216; (see &#8211; avoided significantly &#8230;) that is essential in working out if a trial\u00a0is telling you two treatments really are equivalent, or if the study is just underpowered.<\/p>\n<p>If you&#8217;re designing a trial you&#8217;ll be wanting to be very very sure that this difference, that you&#8217;re gong to base all your study numbers upon is, made upon the best possible grounds. Aren&#8217;t you.<\/p>\n<p><!--more--><\/p>\n<p>Which is why a <a href=\"http:\/\/www.plosmedicine.org\/article\/info:doi\/10.1371\/journal.pmed.1001645\">recent systematic review<\/a> of methods to determine this value &#8211; the DELTA group review &#8211; is really interesting reading. They identified papers which used variants of seven approaches, six of which address the idea of an &#8216;important&#8217; difference:<\/p>\n<ul>\n<li>&#8216;anchoring&#8217; an important difference (patient or expert opinion)<\/li>\n<li>distributional differences (looking for\u00a0changes that are greater than you might find by chance in a group taken at random from the\u00a0population)<\/li>\n<li>health economic approaches (a difference that would make a cost-effective change)<\/li>\n<li>&#8216;better than d&#8217; (assuming that if it makes a standard effect size change of more than 0.2 it&#8217;s a small, meaningful difference)<\/li>\n<\/ul>\n<p>with two also opening to find &#8216;realistic differences as well &#8230;<\/p>\n<ul>\n<li>formal opinion-seeking (via survey +\/- delphi type things &#8211; explicitly requests what might be realistic too)<\/li>\n<li>evidence-based review (looking at what other trials have found to be realistic and regarded as important +\/- put into practice)<\/li>\n<\/ul>\n<p>and\u00a0one which just looks at a &#8216;realistic&#8217; difference<\/p>\n<ul>\n<li>pilot study (demonstrating the differences that you might well find but not commenting on importance)<\/li>\n<\/ul>\n<p>&#8216;Important&#8217; differences are as we discuss &#8211; what matters to patients. &#8216;Realistic&#8217; differences are also vital &#8211; but they are process-vital &#8211; they say what we are likely to realistically find. Ideally both elements would be incorporated, but very few of the published papers they examined explicitly demonstrated how they did this.<\/p>\n<p>Which of these\u00a0is &#8216;right&#8217; is tricky to say. In trial design, a realistic and important difference seems essential. In\u00a0taking trial findings forward, it would be great to see how the\u00a0researchers had found out why the difference they are seeking is meaningful. It&#8217;s all good food for thought though.<\/p>\n<p>&#8211; Archi<!--TrendMD v2.4.8--><\/p>\n","protected":false},"excerpt":{"rendered":"<p>We&#8217;re great fans in the Archimedes blog of trying to get people to think about the meanings and impacts of research, like asking What would Jack want and not believing p-values. One key idea is that of an &#8216;important clinical difference&#8216; (see &#8211; avoided significantly &#8230;) that is essential in working out if a trial\u00a0is [&#8230;]<\/p>\n<p><a class=\"btn btn-secondary understrap-read-more-link\" href=\"https:\/\/stg-blogs.bmj.com\/adc\/2014\/05\/14\/underpowered-and-over-here\/\">Read More&#8230;<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[79],"tags":[],"class_list":["post-926","post","type-post","status-publish","format-standard","hentry","category-archimedes"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/stg-blogs.bmj.com\/adc\/wp-json\/wp\/v2\/posts\/926","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/stg-blogs.bmj.com\/adc\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/stg-blogs.bmj.com\/adc\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/stg-blogs.bmj.com\/adc\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/stg-blogs.bmj.com\/adc\/wp-json\/wp\/v2\/comments?post=926"}],"version-history":[{"count":0,"href":"https:\/\/stg-blogs.bmj.com\/adc\/wp-json\/wp\/v2\/posts\/926\/revisions"}],"wp:attachment":[{"href":"https:\/\/stg-blogs.bmj.com\/adc\/wp-json\/wp\/v2\/media?parent=926"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/stg-blogs.bmj.com\/adc\/wp-json\/wp\/v2\/categories?post=926"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/stg-blogs.bmj.com\/adc\/wp-json\/wp\/v2\/tags?post=926"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}