Austerity, Climate Science, and Data Sharing

By Peter Bokulich

The controversy surrounding the Reinhart and Rogoff paper is interesting for several reasons. The reason we’re interested in it today, is because it raises some important questions about the ethics of data sharing in science.

Reinhart and Rogoff’s paper claimed to show that since World War II, countries with a debt in excess of 90% of their GDP (gross domestic product) experience significantly lower growth than did countries with lower debts. This claim was then used by several politicians around the globe (including, notably, Paul Ryan as the Republican vice presidential candidate in 2012) to argue that countries should cut back their spending lest they cross this dangerous 90% threshold. This in turn, led to various measures of “austerity” – that is, cuts in spending.

Several economists argued that these pushes for austerity were a huge mistake. John Maynard Keynes argued in the 1930s that when there is a recession or depression, the government should spend more money to get the economy going again. Unlike the private sector, the government can borrow (or even print) money to get people working and increase growth. These economists argued that austerity would make the current economic crisis (caused by the collapse of the mortgage bubble in 2008) even worse.

However, worries about 90% threshold, together with worries about high debt leading to inflation, resulted in powerful political pressure to cut back government spending in America, and even more so in Europe. And a fair bit of the intellectual authority for this pressure came from Reinhart and Rogoff’s paper.

While this was going on, a number of economists tried to replicate Reinhart and Rogoff’s results, but they were unable to do so. Reinhart and Rogoff did not publish their data and calculations, so no one was quite sure how they got the results that they did.

In 2013, Reinhart and Rogoff shared their data and the Excel sheet containing their calculations with a graduate student at UMass Amherst. The student, Thomas Herndon, discovered that the spreadsheet calculations contained an error which made it seem that high-debt countries had lower growth than they actually experienced. Further, it seems that Reinhart and Rogoff made some questionable judgements about which particular data to include, and how it should be weighted.

In short, the more careful analysis that resulted is that there is indeed a correlation between increased debt and lower economic growth, but (a) it isn’t clear two what extent the debt is the cause of the low growth, and to what extent the debt is instead the result of economic recession, and (b) there is no sharp threshold, or cliff, at around 90% that looks importantly different from the general trend that higher debt tends to correlate with lower growth.

Indeed, the 90% number simply came from the fact that Reinhart and Rogoff decided to group countries’ debt level into three bins: below 30% GDP, from 30% to 60%, from 60% to 90%, and 90% and above. Notice that this means that a country with debts of 200% GDP is grouped with a countries with debts at 95% of GDP. Thus a general trend (which might not even reflect a direct causal link) was made to look like a dangerous threshold that should be avoided at all costs.

But a more sober account is that while in general more debt might inhibit growth, there’s no reason to think that 90% there’s anything special about debt levels of 90% GDP. The difference between 80% and 100% isn’t more significant than the difference between 50% and 70%.

Further, when we are in an economic crisis, and only the government has spending ability to pull us out of it, it may well be dangerously counterproductive to try to trim back government spending out of concerns about the debt. (And this is even more true if, as has recently been the case, interest rates are at historic lows.) Many have argued that Reinhart and Rogoff’s paper has needlessly contributed to a great deal of hardship over the past couple years.

Could this have been avoided if Reinhart and Rogoff had posted their data and spreadsheet when they first published their paper?

On the one hand, the demands of open science and replicability seem to imply that scientists (and researchers more generally) should make as much of their data and methods available as they reasonably can. On the other hand, there often seem to be good reasons for scientists to keep their data and methods to themselves.

One such reason is that scientists might legitimately be worried that they might get scooped. If you put a lot of time and energy into producing a set of data, or a particular computational model, you’re going to want to use it to develop your own career and research projects. You don’t want someone else to scoop you by publishing papers with your data.

A second reason is that the more information you make available to the public, the more material political adversaries have to attack you. This can be clearly seen in the case of climate change science. Climate scientists are often wary of making their data and models public because deniers will comb through their work to find any small misstep they can and then try to use that mistake to undermine the scientist’s work as a whole. Of course, there always will be some questionable moves or small mistakes in any human project; and it is indeed part of scientific process to weed out these errors. But the concern is that this scientific good will far be outweighed by the misleading political attacks that will be built on any error that the denialists can find.

The dynamics of this can be seen in the numerous episodes of deniers filing Freedom of Information Act requests for climate scientists’ papers and data, and the efforts by the scientists to keep this information private – or at least to restrict it to the eyes of competent scientists. Likewise, the theft and release of e-mails written by climate scientists indicates demonstrates how deniers try to make mountains out of molehills and turn any personal character flaw into a grand conspiracy.

So. Should scientific data be made public? When? Why?