And what are the methods for testing assumptions that do not (implicitly or explicitly) use statistical significance test reasoning? Eyeballing is not a method.

]]>Bob, who will do the R and jamovi legwork will of course see your discussion, and may jump in. I’ll also make sure that Jonathan, creator of jamovi, sees your comments.

Learning objectives–I do know about these, and (a while ago) have used textbooks that emphasise them. I confess that I’m not a huge fan in practice. They usually seem to me contrived and not necessarily helpful for students, especially when we’re aiming for deep learning. But I haven’t followed the literature on their use these last few years, so maybe now there’s evidence to say that students value them and do better? I’ll certainly check out your link and suggestions.

Thanks again, and all the best for your Christmas break,

Geoff

I had a look into jamovi. At first, I was worried that with the interface/menu approach one would lose reproducibility. But then – looking at the videos – I understood that one can share the whole file with all details or even to use the underlying R code. I was especially impressed by the jamovi video on osf.io.

I understand completely why you decided to wrote the ESCI software and how you could incorporate your “local exerpiments” with jamovi. (With “local experiments I am refering to Donald Schön “The Reflective Practiotioner” and “Educating the Reflective Practitioner”.) I see these local experiments in your book as an essential and outstanding feature! (e.g. “If a data point lies exactly on a bin boundary, does ESCI place it in the bin to the left or the right? Perform an experiment to find out.” or “With the Laptop data …, what two extra data points could you add without changing the mean or the SD?”)

As jamovi follows a more holistic approach – at least concerning essential statistics competences at the undergraduate levels – all the worries with R-programming, looking and installing the right R-packages etc. are not important anymore. (But as jamovi tries also to build a community supported programme development this advantages may disappear in the near future.)

But still I think there are some possible weaknesses of jamovi to overcome:

1) Reporting: From my first experience exporting tables and graphics is a little bit cumbersome and not solved very well. In contrast to R: few formats are available and without R Markdown one loses the possibility of “literate programming”, to combine the report with data analysis and outcome report.

2) Advanced Statistics: I am lacking professional knowledge but I am not sure if jamovi is suitable also for advanced professional statistics. If not (and some remarks in the last jamovi video – Nr. 52: next steps – suggests this) then one has to change with growing experience at some point to R and learn another software with a different interface (RStudio). When and how this change has to occur is from an educational point of view important and has to be planned carefully.

3) In support of jamovi people argue that it is a great solution for people who want to change from the very expansive IBM-SPSS to a cost-free open source solution. Actually I do not understand this argument, as I believe that (a) most people in an introductory statistics course (freshman) do not have previous knowledge on SPSS and (b) there is with PSPP a SPSS-like open source solution.

My personal conclusion so far: I am not sure if it is the best approach to circumvent the steep learning curve of R instead to smooth the many challenges for R beginners. In this sense jamovi would make sense – quasi as an introduction to R using heavily the jmv packages for R. What I had in mind in my first comment was an interactive tutorial using shiny and learnr in order to help beginners to learn R. (I will spend Christmas Holiday for trying to explore this options using your book and approach. Maybe I can show you in January some examples.)

Finally another suggestion for new edition: At the moment you are starting each chapter describing the content covered in this chapter. But instead of using what will be discussed you should formulate the learning outcomes, e.g. the competences student will have after they have successfully learned the content of the chapter. This is a new way of teaching: Important is not anymore was we (as teacher) are planning but what students are able to do after the have successfully learned the material we have offered. I have no experience with Australia but in Europe this new (student oriented) approach is mandatory. (There is a lot of material on “Learning Outcomes” on the internet, you can also have a look at http://valeru.net/en/, a project we have finisehd succesfully last year in order to train Higher Education people of the Russian federation to implement this new approach.

]]>Geoff

PS For a demo, at YouTube search for ‘dance of the p values’ or ‘significance roulette’ ]]>

The experimental details and data should be available to all who are interested and for eventual replication by others. If folks don’t know the limitations of the “p” value, perhaps they are in the wrong business.

I am not a statistician but know enough to invite one into the game during experimental design and evaluation. All publication reviews should include a formal statistical evaluation.

Perhaps results should be accompanied with a “no known independent replications” statement as a reminder.

]]>Thank you for your comment, your highly positive words, and your observations!

Yes, in a second edition we would certainly update, and probably expand, the Open Science material. It’s exciting that OS is progressing so quickly. The OS material in the book was actually written about 2.5 years ago, which in OS terms seems like half a lifetime.

Yes again. R is definitely the way to go. (You may know that there is an R workbook and guide to go with ITNS, at the publisher’s companion site for the book. Not the same as having it integrated all through, I know.) My decision 20 years ago to start building ESCI in Excel was the right decision then, but, happily, the software world has changed massively since. R and its various tools and packages must be top of our list of options, but I think it’s also worth considering JASP and jamovi, both of which are based on R, open source, and provide extensive and expanding data analytic capabilities. They are each intended as better (and cheaper!) replacements for SPSS.

Thanks again for your advice and your enthusiasm. I hope ITNS and its ancillaries can serve you well,

Geoff ]]>

I think for the second edition you could elaborate on these two points:

(a) There was much change in websites servicing Open Science. You could point to these new sites, explain their functionality and give exercises to use it (like to get an ORCID-Number, preregistering a study design, loading and describing a dataset etc.

(b) Constructing knowledge with your exploring approach is done with your ESCI -Excel software. The problem here is that your tool for explaining/exploring is not the same as the tool for using statistics on a professional level. My recommendation would be to use R and to provide all the interactivity of your ESCI – software (and much more) with R, RStudio and Shiny. There is also “learnr” – a program packages for using R as a tutorial software (https://rstudio.github.io/learnr/.

Please keep in mind that I am not a Statistician but from by professional training a sociologist, working as professor in educational technology and training of digital competencies in an Austria university. One of my specialization in the last two years was to work on data literacy education (and Open Science). This new subject area was also the reason that I read many introductory books on statistics and got so excciting about your approach.

]]>Geoff ]]>

The only other thing I can say from my own experience is that the R shiny library is easy to get things running in, but can feel clunky to use, and hosting applications is potentially a problem. One potentially interesting use of shiny would be to offer a package in which some of the functions open small shiny apps to interact with.

]]>Thanks, that’s great. We appreciate the issues with Excel, including the lack of perfect compatability with Mac. Bob has some prototypes of various different approaches to building the simulations. The software decisions are critical, not least because any major change would be a very large task.

BTW, it’s good to see prediction intervals, and essential to have the definition at the bottom. My first inclination is to assume ‘prediction interval’ refers to an interval with a stated probability of including the mean of a replication, rather than a single data value. But, as you know, the term does not have a single agreed usage.

Geoff ]]>