Pictures of uncertainty: Dancing with Pierre in Paris

A while back I wrote a post about Pierre Dragicevic, an HCI researcher in Paris who for years has been working to persuade researchers in his field to adopt better statistical methods.

I wrote about his wonderful talk that presents lots of different dances–not only of means, but p values, and of CIs, dichotomous decisions, and several others things. (At that link, scroll down a little to ‘Materials’ for download of the slides, list of references, and more.) Each dance is a picture of uncertainty or, rather, a movie of how uncertainty is represented for successive samples in a simulation.

Click here to see all the dances in action.

He recently let me know that he’d given the talk again. You can see this latest talk here. At about 4.50, note the great quote from Andrew Gelman:

“Statistics has been described as the science of uncertainty. But, paradoxically, statistical methods are often used to create a sense of certainty where none should exist.”

That’s the heart of the NHST vs estimation question: a p value can easily give a seductive but illusory sense of certainty, but a CI puts the extent of uncertainty in our faces.

After watching the talk again, I mentioned to him that he might have brought out more strongly the advantage of CIs over p values that a CI gives us a useful indication of its dance–CI length indicates the general width of the dance–whereas a p values says almost nothing about the dance from which it comes. He agreed, but noted that the implications of dances need to be learned and, in his experience, students can have difficulty building good intuitions about dances.

That’s a good point. In ITNS Bob and I discuss CI dances, and interpretation of a CI in terms of how its length tells about what’s likely to happen on replication. We have found this approach to work very well with students, but I’d love to see empirical investigations of how effective our approach to teaching the new statistics is, in practice. Anyone up for the challenge of doing that?

Geoff