JAMA Opposes p<.05 Decision Making

A recent viewpoint article (free download) in JAMA argues that decisions should not be based on mere p value criteria, but need “consideration of the outcome in terms of effect size and accompanying CIs, [and] placing the findings from the trial in the context of the totality of the existing relevant evidence”. Hooray for JAMA! The overview:

JAMA. Published online May 8, 2020. doi:10.1001/jama.2020.3508 

A common argument given in defence of NHST is that (1) in the real world we need decisions, (2) NHST and p<.05 is a way to make such clear decisions that is (3) based on the evidence and (4) objective. Yes, 1 and 2 are true, but 3 is only partly true, and 4 may be true if all details of the data analysis and decision procedure are preregistered. However, a p value reflects N as much as anything, and, most importantly, other vital factors need to be considered in making a real-world decision, beyond the current data. Decisions need to be informed by data, of course, preferably via confidence intervals and meta-analysis. But they also should reflect consideration of alternatives, costs and benefits, the values of relevant parties, and so on. ‘Conclusions’ above acknowledges all this. Hooray for JAMA!

That’s all fine, but the title and main consideration of the viewpoint are surprising: What about non-statistically significant results? Given the recommended approach to decision making, the viewpoint argues that there may even be cases in which such a non-sig result might justify a change in clinical practice. Phew–it seems a very forced argument to me: Take some very weak evidence and dream up a case in which that might just tip the balance and lead us to change practice? Perhaps, but …

The examples discussed, however, emphasise the role of prior evidence (meta-analysis is not mentioned explicitly) and of costs and benefits. In addition, the extent of uncertainty, even if a clinical decision has to be made, needs emphasis. So that’s all good.

The article includes this link to an interview by Howard Bauchner, JAMA’s Editor in Chief, with Paul Young, author of the viewpoint. OK, it’s quicker to skim the article than to listen to the interview, but the interview makes clear that Bauchner takes seriously the need to move on from p<.05. That, for me, is the main point. Hooray for JAMA!


P.S. Many thanks to Anoop Balachandran for the heads up.


Leave a Reply

Your email address will not be published. Required fields are marked *