Get Started

Here are some handy resources to get you started with the New Statistics

Learning

• Of course, we think Introduction to the New Statistics is a great source for learning the New Statistics approach.  But there are lots of other really good resources out there.
• To learn the Bayesian approach to the New Statistics (which we heartily recommend):
  • (J. K. Kruschke & Liddell, 2017)
  • Doing Bayesian Data Analysis, 2nd edition (J. Kruschke K., 2014)
• Learn about estimation from one of the masters:
  • Statistics with Confidence (Smithson, 2000)
  • Confidence Intervals (Smithson, 2003)
• And here are some helpful papers on understanding effect sizes and/or meta-analysis:
  • A great primer on effect sizes, the subtle differences between effect sizes for different designs, etc:  (Lakens, 2013)
  • An excellent tutorial on effect sizes and how to report them (Pek & Flora, 2017)
  • The bible of effect sizes and meta-analysis: (Cooper, Hedges, & Valentine, 2009)

Software:

• ESCI is a free set of Excel spreadsheets that can help provide New Statistics analysis and figures for many common research designs.  It is available for download directly from this website: ESCI
• ESPSS is a free set of plugins for SPSS to provide New Statistics output for common research designs.  This project is still early in development, but you can find modules for 2-group designs and correlations.  The GitHub page is here: https://github.com/rcalinjageman/ESPSS
• R is the Swiss Army Knife of statistics, and you can certainly accomplish New Statistics analysis with R.  It can be a bit tricky, though, to get R to do all you’d like.  Fortunately, David Erceg-Hurn has put together a guide to the New Statistics in R.  You can download the guide here: https://osf.io/rmn8p/
  • The ITNS package by David Erceg-Hurn can make the cool plots that show a difference axis, a nice way to show the effect size of interest for a two-group comparison
  • The userfriendlyscience package in R has lots of cool functions for New Statistics analysis and outpu
• Jamovi is a promising free, open-source statistics program that provides a user-friendly interface for R.  We don’t have any specific New Stats commands for Jamovi yet, but hope to develop some soon.
• Jasp is a free, open source statistics program with an emphasis on Bayesian analysis.  It will produce credible intervals for many types of designs, the Baysian version of a confidence interval.

Sample-Size Planning (Planning for Precision)

• ESCI provides worksheets for planning two-group designs for precision
• Some other good planning for precision tools are:
  • The userfriendlyscience package in R
  • This great tool for complex designs by Gerben Mulder
• For the Bayesian approach, see Doing Bayesian Data Analysis (J. Kruschke K., 2014)
• There is a fantastic set of articles on planning for precision by Ken Kelley and associates
  • A general overview of the approach (Maxwell, Kelley, & Rausch, 2008)
  • For regression analysis (Kelley & Maxwell, 2003)
  • For mediator analysis (Kelley, 2007)

Meta-Analysis:

• ESCI provides some pretty good tools for meta-analysis.  It enables meta-analysis of raw scores or standardized effect sizes.  It enables subgroup analysis.
• Open Meta Analyst is a free, open source GUI for the metafor package.  It provides meta-analysis for several different types of designs, subgroup analysis, and even meta-regression.  It’s pretty easy to use, though it doesn’t seem to be in development any longer.
• Meta Essentials is a free Excel workbook that makes many types of meta-analysis straightforward.
• The metafor package for R is amazing.  It’s not very hard to use, and there is a very lively listserv where you can pose questions.  It’s worth just rolling up your sleeves and learning how to use it.

Open Science:

• The Open Science Framework is your one-stop shop for pre-registration, data sharing, publishing your file drawer and more.
  • The Open Science Framework has developed a pre-registration template that walks you through the pre-registration process. You can download this template here.
  • If you’d like to see what a completed template looks like, here is an example that plans a replication of a study from Ottati et al. (2015).
  • At of 2017 you can also win $1,000 for pre-registering a study with OSF and then publishing the study in a participating journal.
• Some of the initiatives towards Open Science going on in cognitive neuroscience (Gilmore, Diaz, Wyble, & Yarkoni, 2017)

Copy of ESCI for IU demo: ESCI-intro-chapters-10-16-IU