Category: Replication

Replication: ‘Psychological Science’ does the right thing

I have been enjoying Bob’s series of posts about replication. (Go to our home page and scroll down to see links and a few lines of text about each of the 5 posts, with title starting ‘Adventures in Replication’.) Actually,

Posted in Open Science, Replication, The New Statistics

Adventures in Replication – Reviewers don’t want to believe disappointing replication results

Trying to publish replication results is difficult.  Even when the original evidence is very weak or uncertain, reviewers tend to look for reasons to explain away a smaller effect in the replication.  If nothing comes to mind, reviewers may even

Posted in NHST, Replication, The New Statistics

Brain Stimulation – Can we trust the empirical record?

Brain stimulation research has been exploding in neuroscience.  First came the rapid adoption of Transcranial Magnetic Stimulation (TMS), a technique in which powerful magnetic fields are used to create inductive currents within the skull.  More recently, Direct Current Stimulation (DCS)

Posted in Applied research, Open Science, Replication

Adventures in Replication: p values and Illusions of Incompatibility

Here’s an idea I run into a lot in peer reviews of replication studies: If the original study found p < .05 but the replication found p > .05, then the results are incompatible and additional research is needed to

Posted in ITNS, NHST, Replication, Statistical graphics, The New Statistics

Adventures in Replication: Your replication appears to be somewhat underpowered

Many journals now proclaim their openness to replication research.  Behind the scenes, though, replication manuscripts are often met with impossible demands and/or insane double-standards. Here’s an example from an editor at a prominent social psychology journal: the studies appear to

Posted in Replication, The New Statistics

Adventures in Replication: Scientific journals are not scientific

The essence of science is seeking and weighing evidence on both sides of a proposition.  One might think, then, that when a scientific journal publishes a research paper it then acquires a special interest in publishing subsequent replications or commentary

Posted in Replication

Adventures in Replication: Introduction

Over the past 5 years or so, I (Bob) have been a bit replication crazy–I’ve conducted about 10 direct replication projects in collaboration with undergraduate students at Dominican.  I became obsessed in part because I wanted to know for myself

Posted in Replication

Red, Romance, and Replication

I have a new replication paper out today, a collaboration with DU student Elle Lehmann (Lehmann & Calin-Jageman, 2017) .  The OSF page for the paper with all the materials and data is here: (Calin-Jageman & Lehmann, 2015) The

Posted in Open Science, Replication

Pictures of uncertainty: Dancing with Pierre in Paris

A while back I wrote a post about Pierre Dragicevic, an HCI researcher in Paris who for years has been working to persuade researchers in his field to adopt better statistical methods. I wrote about his wonderful talk that presents

Posted in ITNS, Open Science, Replication, Statistical graphics, Teaching

Danny Kahneman: From p values to Nobel Prize

You meet a red-headed person who is a bit short-tempered then, later, another who is similarly touchy. You start to believe that red hair signals ‘watch out’. Really? You are leaping to a conclusion from an extremely small sample! But

Posted in ITNS, NHST, Replication