Category: NHST

p values and outrageous results

If you were researching a muscle-building supplement and read that a test of the supplement produced an increase in muscle mass by 200% within a month, you’d be right to be skeptical.  Perhaps randomization had broken down, perhaps there was

Posted in NHST

Danny Kahneman: From p values to Nobel Prize

You meet a red-headed person who is a bit short-tempered then, later, another who is similarly touchy. You start to believe that red hair signals ‘watch out’. Really? You are leaping to a conclusion from an extremely small sample! But

Posted in ITNS, NHST, Replication

From NHST to the New Statistics — How do we get there?

APS just wrapped up.  Geoff and I were privileges to help host a symposium on making progress moving the field away from p values towards the New Statistics.  Our co-conspirators were fellow text-book author Susan Nolan, Psychological Science editor Stephen

Posted in NHST, Open Science, Teaching, The New Statistics

The persistence of NHST: “Wilfully stupid”?

I recently gave a research talk to Psychology at La Trobe, my old University–although I now live an hour out of the city and rarely visit the campus. I decided to turn things around from my previous few talks: Instead

Posted in NHST, Replication, The New Statistics

p intervals: Replicate and p is likely to be *very* different!

The Significance Roulette videos (here and here) are based on the probability distribution of the p value, in various situations. There’s more to the second video than I mentioned in my recent post about it. The video pictures the distribution of replication

Posted in NHST, Replication

Significance Roulette 2

In my post of a couple of days ago I gave the link to Significance Roulette 1, a video that explains how to generate the roulette wheel for a ‘typical experiment’, by which I meant an independent groups experiment, N = 32

Posted in ITNS, NHST, Replication, The New Statistics

Significance Roulette 1

If you run an experiment, obtain p = .05, then repeat the experiment–exactly the same but with a new sample–what p value are you likely to get? The answer, surprisingly, is just about any value! In other words, the sampling

Posted in ITNS, NHST, Replication, The New Statistics

A conference about–wait for it–the p value! But other things too.

In March 2016 the American Statistical Association (ASA) posted online a policy statement about the p value. You can see it here. This was remarkable–for one thing because it was the first time the ASA had made a public pronouncement

Posted in NHST, Open Science, The New Statistics

Wise words from Ken Rothman, who is statistical reform royalty

I (Geoff) recently came across an article published in 2014 with the title Six Persistent Research Misconceptions. All six are important, but it’s no. 6 that would be most familiar to anyone reading ITNS: Misconception 6. Significance testing is useful

Posted in NHST, The New Statistics

p Hacking: More than you ever wish to know

I recently received an email telling me that an article I had reviewed for a journal had achieved 10,000 views. The astonishing thing was that the email arrived less than 3 weeks after the article had been published online! Believe

Posted in NHST, Open Science