The ASA and p Values: Here We Go Again

The above announcement is from the February ASA (American Statistical Association) newsletter. (See p. 7 for the announcement and the list of 15 members of the Task Force.)

Why won’t statistical significance simply whither and die, taking p<.05 and maybe even p values with it? The ASA needs a Task Force on Statistical Inference and Open Science, not one that has its eye firmly in the rear view mirror, gazing back at .05 and significance and other such relics.

I shouldn’t be so negative: I definitely am glad that ‘reproducibility’ is a focus, even if ‘Open Science’ may suggest a wider view.

To welcome the new Task Force, Andrew Gelman posted an invitation to discussion. His post, which is here, is sensible. Stuart Hurlbert makes some useful early contributions, but most of the 152 (as of now) comments make me tired and depressed as I skim through.

You may recall the background, including:

I suggest that much of what’s needed can be summarised by this basic logic:

  • We need Open Science (replication, preregistration, open data and materials, fully detailed publication whatever the results, …).
  • Open Science requires replication.
  • Replication requires meta-analysis.
  • Meta-analysis requires estimation. (Frequentist, Bayesian, or…)

Note that there is NO necessary role for NHST or p values in any of the above. Historically, p values have caused much damage, especially by prompting selective reporting, which biases meta-analysis, perhaps drastically. Simply let them whither and fade into the background.

Beyond all that, we’d love to be moving to more quantitative modelling, which makes estimation even more necessary.

Watch for a call for submissions to the Task Force.

Bring on the revolution…


Leave a Reply

Your email address will not be published. Required fields are marked *