The inertia of poor practices: Updated reporting requirements don’t have much impact at JNeuroPhys

Here’s a depressing new paper that surveys reporting practices in The Journal of Neurophysiology after it instituted new reporting guidelines for authors ​(Héroux et al., 2023)​. Why depressing? Because the updated guidelines seem to have done little to improve reporting, and this was true even for practices the editors had said would be required. Was that because the editors went to far, asking authors to jump through arduous new hoops? No. About 2/3 of authors defied (with impunity) a requirement to consistently report the sample size and statistical test used in their figure captions. Really? Yes, really. And, in general, poor statistical practices abound and have shown little change: about 60% of recent papers interpreted non-significant results as trending towards significance, reporting of exact p values was sporadic, and reporting of effect sizes with confidence intervals has increased but remains rare (30%).

This makes me (Bob) sad. The Journal of Neurophysiology is a strong journal that is published by a renowned professional society (just check out these beauties: ​(Calin-Jageman, Tunstall, Mensh, Katz, & Frost, 2007)​ and ​(Sakurai, Calin-Jageman, & Katz, 2007)​. The editors have clearly worked to take steps to improve statistical reporting, with updated policies in 2016 and 2018. None of the new requirements were especially out there, either — this was a pretty conservative effort for reform. Even in this conducive context, it is clear that the inertia of poor practices is overwhelming — both from authors who won’t adapt and reviewers/editors unequipped and/or unmotivated to consistently appy new policies.

I guess this is only surprising if you’re not yet fully cynical. Blech.

  1. Calin-Jageman, R. J., Tunstall, M. J., Mensh, B. D., Katz, P. S., & Frost, W. N. (2007). Parameter Space Analysis Suggests Multi-Site Plasticity Contributes to Motor Pattern Initiation inTritonia. American Physiological Society. doi: 10.1152/jn.00572.2007
  2. Héroux, M., Diong, J., Bye, E., Fisher, G., Robertson, L., Butler, A., & Gandevia, S. (2023). Poor statistical reporting, inadequate data presentation and spin persist despite Journal awareness and updated Information for Authors. F1000 Research Ltd. doi: 10.12688/f1000research.142841.1
  3. Sakurai, A., Calin-Jageman, R. J., & Katz, P. S. (2007). Potentiation Phase of Spike Timing-Dependent Neuromodulation by a Serotonergic Interneuron Involves an Increase in the Fraction of Transmitter Release. American Physiological Society. doi: 10.1152/jn.00702.2007

I'm a teacher, researcher, and gadfly of neuroscience. My research interests are in the neural basis of learning and memory, the history of neuroscience, computational neuroscience, bibliometrics, and the philosophy of science. I teach courses in neuroscience, statistics, research methods, learning and memory, and happiness. In my spare time I'm usually tinkering with computers, writing programs, or playing ice hockey.

1 Comment on “The inertia of poor practices: Updated reporting requirements don’t have much impact at JNeuroPhys

  1. Hi, Bob. That is disappointing but not wildly surprising. It seems clear to me that journals need to invest in efforts to support authors to use better practices. More specifically, they need in-house stats/methods advisors who check on these things and give authors feedback. Given the high profits of many journals, this should be feasible. Providing such services would improve the quality and hence the value of the publications.

Leave a Reply

Your email address will not be published. Required fields are marked *