The Myth (?) of the Lucky Golf Ball Lives On, Alas

The APS has just given a kick along to what’s most likely a myth: The Lucky Golf Ball. Alas!

Golf.com recently ran a story titled ‘Lucky’ golf items might actually work, according to study. The story told of Tiger Woods sinking a very long putt to send the U.S. Open to a playoff. “That day, Tiger had two lucky charms in-play: His Tiger headcover, and his legendary red shirt.”

The story cited Damisch et al. (2010), published in Psychological Science, as evidence the lucky charms may have contributed to the miraculous putt success.

Laudably, the APS highlights public mentions of research published in its journals. It posted this summary of the Golf.com story, and included it (‘Our science in the news’) in the latest weekly email to members.

However, this was a misfire, because the Damisch results have failed to replicate, and the pattern of results has prompted criticism of the work. Read on…

The original Lucky Golf Ball study

Damisch et al. reported a study in which students in the experimental group were told—with some ceremony—that they were using the lucky golf ball; those in the control group were not. Mean performance was 6.4 of 10 putts holed for the experimental group, and 4.8 for controls—a remarkable difference of d = 0.81 [0.05, 1.60]. (See ITNS, p. 171.) Two further studies using different luck manipulations gave similar results.

The replications

Bob and colleague Tracy Caldwell (Calin-Jageman & Caldwell, 2014) carried out two large preregistered close replications of the Damisch golf ball study. Lysann Damisch kindly assisted them make the replications as similar as possible to the original study. Both replications found effects close to zero.

Meta-analysis of original and replication studies

Here’s Figure 9.8 from ITNS, a forest plot that shows results from six studies from the Damisch group (red), and the two Calin-Jageman & Caldwell replications (blue). The first of the red studies and both of the blue used the lucky golf ball task.

The clear difference between the overall red mean and overall blue mean is shown at the bottom on a Difference axis; it’s -0.77 [-1.15, -0.38], so a clear failure to replicate.

Not replicable, but citable

That’s the title of a 2018 post by Bob lamenting the common pattern of a striking original finding that continues to make waves, even while strong counter evidence from replications languishes in the shadows.

Below I’ve updated his figure showing citation counts for the original Damisch article and the Bob & Tracy replication article. The pattern has not improved these last three years!

The pattern of Damisch results

The six red CIs in the forest plot are astonishingly consistent, all with p values a little below .05. Greg Francis, in this 2016 post, summarised several analyses of the patterns of results in the original Damisch article. All, including p-curve analysis, provided evidence that the reported results most likely had been p-hacked or selected in some way.

Another failure to replicate

Dickhäuser et al. (2020) reported two high-powered preregistered replications of a different one of the original Damisch studies, in which participants solved anagrams. Both found effects close to zero.

All in all, there’s little evidence for the lucky golf ball. APS should skip any mention of the effect.

What next?

Open Science practices will help.  Perhaps high quality replication articles can be marked with big badges and trumpet fanfares? With everything online, it should be possible to add annotations to original articles and provide links to later replications, no doubt with original authors having a right of reply. Meanwhile, we need to keep up the skepticism and eternal vigilance.

Geoff

Calin-Jageman, R. J., & Caldwell, T. L. (2014). Replication of the Superstition and Performance Study by Damisch, Stoberock, and Mussweiler (2010). Social Psychology, 45(3), 239–245. https://doi/10.1027/1864-9335/a000190

Damisch, L., Stoberock, B., & Mussweiler, T. (2010). Keep Your Fingers Crossed! Psychological Science, 21(7), 1014–1020. https://doi/10.1177/0956797610372631

Dickhäuser, O., Heinze, A., Hamm, M. L., Bales, A. S., Bellmann, S. A., Böger, D., et al. (2020). Zwei teststarke, präregistrierte Replikationsstudien zum Einfluss von Glück auf kognitive Leistung (Two high-powered preregistered replication studies on effects of superstition on cognitive performance). Zeitschrift für Pädagogische Psychologie, 34, 51-60. https://doi.org/10.1024/1010-0652/a000263