Bushfires and Open Science: A View From Australia

Our Family Summer in a Time of Fires

We’re just back home after a couple of weeks at the big old family beach house. We had one stinking hot day, 40+ degrees, but, strangely, other days were cool to cold, usually with strong swirly winds. So different from the long spells of searing heat further out to the East. We had only a few beach visits but lots of indoor games and gang self-entertainment by the kids. People came and went, but usually it was a pleasantly chaotic mob of 15 or so people. We watched the Test cricket–beating the Kiwis, yay! And, like the rest of the world, were aghast to see the pictures and hear the reports of the enormous fires up and down the East and South-East coasts.

We were at Anglesea, one of a string of small towns along the Great Ocean Road to the west of Melbourne. There have been big fires down that way in the past, but this year, so far, nothing major, although the peak months of Feb and March are still ahead. We not only enjoyed a family holiday, but–new these last couple of years–also kept a careful eye on the sky to the West, and on the excellent emergency app that pings when any warning is issued for any ‘watch zone’ you care to nominate. So we kept our phones charged and didn’t forget to check the car had a full tank, and drinking water and blankets on board–and we reminded ourselves where the two evacuation areas in the town are, and how we can most quickly get there. And what we musn’t forget if we have to load and leave quickly. Note to self: Find the little old battery radio, to keep nearby to hear emergency warnings if the electricity and phone reception both die.

We drove home through smoke haze, two hours, headlights on, never quite adapting to the stinging smell of smoke. The smoke came, we were told, not from the Victorian fires a few hundred km to the East, but from fires in Tasmania a thousand km to the South. Some of the fires have been burning for months, especially up north, and Sydney has had days of dangerous smoke at various times these last several months. Most ominously, it seems that these fires are on track to increase Australia’s already scandalously high carbon emissions by more than 50% over this July-to-June period, and very likely by 100% or more. More than double! A tipping point, anyone?

Yes, life on this planet is changing and we’re all in it together. But what has this to do with Open Science?I think we can learn by listening to climate scientists–not only about the carbon emission reductions we should have been doing 30 years ago and must, with great urgency, be doing now, but also about what good research practice looks like, under pressures that most of us can only imagine.

BTW, I’ll mention The Conversation (‘Academic rigour, journalistic flair’), which is running what seem to me an outstanding series of pieces on the fires. This week, see the Mon, Tues, Wed, and today’s offerings.

Climate Scientists at Work–Implications for Open Science

The best I’ve read recently is Gergis, J. (2018) Sunburnt Country. Carlton, Vic, Australia: Melbourne University Press.

In the acknowledgements, the author first thanks the teachers in the professional writing course she undertook before writing the book. The training shows–her writing is beautiful and compelling. Gergis tells the fascinating story of her research on the history of climate in Australia, going back centuries and more. She draws on data series from ice cores, tree growth rings, and coral growth rings, as well as evidence from early indigenous people and European explorers. She assembles overwhelming evidence that Australia is getting hotter and drier and, what’s more, is now experiencing weather extremes far beyond natural variability. And that human activities over the last century and more are by far the most important cause of global heating.

Comparable analyses, bringing together a wide range of data and applying a number of climate models, have been carried out for the Northern Hemisphere, but the work of Gergis and her many colleagues is a first for the Southern Hemisphere. The stories in the two hemispheres are roughly parallel, yet different, most notably by showing evidence that the toxic effect of industrialisation started later down south, not until about the mid-19th-century.

I’ll mention several aspects of this–and much other–climate research:

Integration of evidence–similar measures

We are becoming used to using meta-analysis to integrate results based on similar studies using the same or similar measures. Climate scientists do this routinely, sometimes on a very large scale as when different time series of e.g. CO2 concentration are integrated.

Integration of evidence–diverse measures

We sometimes talk about converging approaches as providing a powerful strategy for establishing a robust finding. See for example pp. 414-416 in ITNS. Climate scientists do this on a massive scale, as when multiple data series of quite different types are brought together to build an overall picture of change over time.

Using multiple quantitative models

Some fields in psychology use quantitative modelling, and its use is spreading slowly across the discipline. In stark contrast, the development of highly complex quantitative models is core business for climate researchers. Psychologists might feel that human behaviour, cognition, and feelings are about as complex as anything can get, but climate scientists are attempting to model and understand a planet’s biosphere, whose complexity is approaching comparable, I think. One of our arguments for the new statistics is that estimation is essential for quantitative modelling, so using estimation is an essential step towards a quantitative discipline. Gergis and her team apply multiple climate models to their diverse data sets to account for what has happened in the past and provide believable predictions for the future. You can guess that these predictions are scary beyond belief, unless our practices change drastically and very soon.

Open data and analysis code

It seems to be taken for granted that data sets are openly available, at least to all other researchers. The same for the models themselves, and the analyses carried out by any research group.

Reproducibility and peer scrutiny

Gergis describes how, again and again, any analysis or evaluation of a model by her group is subjected to repetition and intense scrutiny, at first by others within the group, then by other researchers. Only after all issues have been dealt with, and everything repeated and re-examined once more, is a manuscript submitted for publication. Which leads to further exacting peer scrutiny, and possible revision, before publication. It’s exhausting and sobering to read about such a painstaking process.

The Dark Side

You probably know about the hideous harassment meted out to climate researchers. Any aspect of their work can be attacked, with little or no justification. Vitriolic attacks can be personal. They may have to spend vast time and emotional energy responding to meaningless legal or other challenges. This toxic environment is no doubt one reason for the intense scrutiny I described above. We know all that, but even so it’s moving and enraging to read the trials that Gergis and her group have endured.

Large is Good? Small is Good?

Discussions in psychology of p-hacking and cherry-picking almost always assume that large is good: Researchers yearn for effects sufficiently large to be interesting and to gain publication. Open Science stipulates multiple strategies to counter such bias. Climate science offers an interesting contrast: Climate scientists tend to scrutinise their analyses and results, dearly hoping that they have slipped up somewhere and that their estimates of effects are too large. Surely things can’t be this bad? So back they go, checking and double-checking, if anything making their analyses and conclusions more conservative, rather than–as perhaps in psychology–exaggerated.

Researchers Have Emotions too

Of course many psychologists are emotionally committed to their research–we can feel disappointment, frustration, and–with luck–elation. I suspect, however, that rarely are these emotions likely to match the strength of emotions that climate scientists sometimes experience. Besides the personal cost of the vitriolic personal attacks, finding results that map out a hideous future for humankind–including our own children and grandchildren–can be devastating. In a forceful opinion piece a few months ago (The terrible truth of climate change), Gergis wrote:

“Increasingly after my speaking events, I catch myself unexpectedly weeping in my hotel room or on flights home. Every now and then, the reality of what the science is saying manages to thaw the emotionally frozen part of myself I need to maintain to do my job. In those moments, what surfaces is pure grief.”

And so…

All the above may make our Open Science efforts pale. But, as we try to figure out how to improve the trustworthiness of our own research, I think it’s worth pondering the strategies this other discipline has adopted in its own effort to give us results that we simply must believe.

Geoff

P.S. A warm thank you to those friends and colleagues near and far who have sent enquiries, and messages of concern and support. Very much appreciated. Yes, we are all in this together.