Danny Kahneman: From p values to Nobel Prize

You meet a red-headed person who is a bit short-tempered then, later, another who is similarly touchy. You start to believe that red hair signals ‘watch out’. Really? You are leaping to a conclusion from an extremely small sample! But humans seem to have a strong tendency to draw conclusions from small samples–we tend to believe that even a tiny sample is likely to closely resemble the population it came from.

This ‘law’–actually a fallacy–was described in the classic 1971 article by Amos Tversky and Daniel Kahneman. This article presented evidence that even research psychologists trained in mathematics and statistics often tend to severely underestimate the extent of sampling variabiity.

A related finding was that they tend to greatly overestimate how close the result of a replication experiment is likely to be to the original experiment. Sound familiar? Decades later, the dance of the p values and significance roulette convey versions of the same message–especially in relation to p values. Replicate, and the results may well be surprisingly different, alas–especially if p values are involved.

I mention this because I’ve just finished reading The Undoing Project: A Friendship That Changed Our Minds, by Michael Lewis. You may know some of Lewis’s earlier books, perhaps Moneyball, The Big Short, or Flashboys.

In Undoing, Lewis tells the story of Amos Tversky and Danny Kahneman, two brilliant minds and two very different people. They talked and laughed for days and days and decades, dreamed up novel experiments, and identified all sorts of strange ways that people reason and make decisions. They also fought for Israel in several wars, their most important contributions coming from their psychological insights.

It’s a gripping tale, well told, and an insight into one highly successful way to do creative and collaborative science. After the law of small numbers came behavioral economics and in 2002, sadly after Tversky’s death, the Nobel Prize in economics.

In ITNS (pp 359-360) we discuss the example of the flying instructor who gave praise for a smooth landing by a trainee pilot, and harsh words for a rough landing. Regression to the mean can explain what the instructor observed: after praise, most often a less good landing, but after harsh words quite likely a better landing. Lewis explained that this example came straight from Kahneman’s observation of pilot training in the Israeli air force in the early days. Kahneman’s insight that regression to the mean could account for the puzzling observations led to changes in the way pilots were trained.

Lewis’s book could use some editing, and you can easily skip Chapter 1, but even so it’s a great read. You can get a flavor from this article in The New Yorker by Cass Sunstein and Richard Thaler–which is definitely worth a look.

Enjoy…
Geoff

P.S. Personal note. Kahneman is married to Anne Treisman, herself a distinguished psychologist, who was my doctoral advisor in Experimental Psychology at Oxford.

Lewis, M. (2016). The undoing project. A friendship that changed our minds. W. W. Norton. ISBN 978-0-393-25459-4
Tversky, A., & Kahneman, D. (1971). Belief in the law of small numbers. Psychological Bulletin, 76, 105-110.