Moving to a World Beyond “p < 0.05”
The 43 articles in The American Statistician discussing what researchers should do in a “post p<.05” world are now online. See here for a list of them all, with links to each article.
The collection starts with an editorial:
Go here to get the full editorial as a pdf.
Bob and I commented on earlier drafts of the editorial, as did some other authors of articles in the collection. I think the published version is great, even if, as usual, I’d like it to have gone a bit further towards virtually always ending the use of p values. But there are very welcome strong recommendations for the use of estimation, as well as many other wise words.
We’re pleased that the authors of the editorial elected to refer to our article (more on that below) in 5 places, and to quote our words in 4 of those (see pp. 3 (twice), 9, and 10 in the pdf). A very strong theme of the editorial is that researchers should always ’embrace uncertainty’, which is a major theme also in our article, among others.
Go here for the pdf of our article,
Yes, my name is listed as a co-author, but Bob wrote the article, and an excellent job he did of it too! If all researchers followed our (his) advice, the world would be a much better place–says he modestly. But have a squiz and see what you think.
The editorial includes (pp. 10-18 in the pdf) a brief dot point summary of each article. The summary we contributed of ours is:
1. Ask quantitative questions and give quantitative answers.
2. Countenance uncertainty in all statistical conclusions, seeking ways to quantify, visualize, and interpret the potential for error.
3. Seek replication, and use quantitative methods to synthesize across data sets as a matter of course.
4. Use Open Science practices to enhance the trustworthiness of research results.
5. Avoid, wherever possible, any use of p values or NHST.
Here’s to the onward march of Open Science and better statistical practice!
Geoff
Thanks Anoop. Yes indeed!
I think this is the most important point “Seek replication, and use quantitative methods to synthesize across data sets as a matter of course.”
You can have the greatest effect size and very narrow CI from your study, but it may not mean much when you combine the data from other similar studies, like a meta-analytic approach. And no decision should be ever be made from a single study. You have to look at the totality of the evidence,the risk, economics and so forth. If people understood this uncertainty in a single study no matter how well it is done, there would be less calls for why we need to categorize decisions using a p-value in the first place.
Great work both of you!