Should you Trust the PDSI?

In this post, we take a deeper look at the Palmer Drought Severity Index (PDSI), and the extent to which it is a reliable indicator of drought. There are three main sources for this post:

• a 2004 paper by Dai, Trenberth, and Qian (A Global Dataset of Palmer Drought Severity Index for 1870–2002: Relationship with Soil Moisture and Effects of Surface Warming) which introduced their global PDSI dataset, and examined how much fidelity it had to other indicators of drought around the world.
• a 1984 paper by William Alley (The Palmer Drought Severity Index: Limitations and Assumptions) which is essentially a critique of the limitations of the PDSI, but also includes a detailed description of how it is computed.
• However, to really understand the PDSI, you pretty much have to go back to Palmer's original 1965 report, which is fairly clearly written, but nonetheless is 50 pages of closely argued and sometimes ideosyncratic reasoning.

I don't think it's going to be helpful to readers to reproduce all the math, but let me try to sketch at a high level what the PDSI is trying to do.

At its heart, it's trying to track additions to and subtractions from the amount of soil moisture in any given month.  You can imagine that in a column of X feet of soil, at any given time, there is Y inches of water - defined as the number of inches of water that could be removed if the soil were to be completely dried out in an oven and the water recondensed in a measuring cylinder.  If it rains Z inches in a given month, then you might expect that Z will be added to the Y that was already in the soil.  But clearly there is more to it than this.

For example, suppose the soil was almost saturated with water (ie, near what soil scientists call field capacity), and that Z was enough to cause it be completely saturated and then some.  So the PDSI algorithm assumes the excess will then run-off and not be in the soil.  The treatment of run-off (like many things) is quite simplistic - it doesn't allow for the top layer of soil being saturated and running-off, while the lower layers have not yet saturated (due to a heavy storm after a dry period, for example).  Nor does it allow for things like the bottom of the sloping field being saturated while the top is not - fields are assumed to be flat.  The possibility of snow accumulating on the surface is completely ignored (snow lying on top of saturated soil is effectively treated as run-off, which I suppose might be largely accurate except in the timing).  So it's most accurate on flat fields in the presence of relatively gentle rainfalls.

Besides rain, the other biggest term in the soil water budget is evapotranspiration - how much water evaporates off the soil surface, or the surfaces of plants growing in that soil.  On a month of long hot days, this can be huge.  The original PDSI used a simple empirical equation for this called the Thornthwaite equation that only requires knowing the temperature and latitude (the latter to get the day length).  More modern versions use something call the Penman-Monteith equation which is more sophisticated and incorporates information about wind-speed and the degree of solar radiation.  In addition to these equations (which indicate how much water would evaporate from saturated soil), the PDSI assumes that the actual evapotranspiration will be proportional to the amount of water in the subsoil.  The algorithm separately models the topsoil and subsoil.  The topsoil is arbitrarily assumed to be the amount of soil required to hold one inch of water, and the subsoil is everything below that.  The topsoil is assumed to quickly give up all water it has, but the subsoil is more resistant to drying out and the amount of loss to evapotranspiration is assumed proportional to how much moisture is down there.  (Thus in the face of a persistent moisture deficit, the subsoil will dry out with an exponential decline in the amount of moisture).

So all this modeling is somewhat rough and ready, but not crazy.  And it's designed to be something that can be computed from widely observed climatological values alone (in the original version, all that is really needed is the average temperature and precipitation, things widely measured in weather stations).

The next part of the algorithm has to do with noting that soil goes through a normal annual cycle.  In most places, in the winter, there is much less evapotranspiration and the soil tends to fill up with water and then start running-off.  In the summer, there may be more or less rain, but there is almost certain to be much more evapotranspiration, and the soil is apt to dry out and run-off will stop (and streams will dry up and rivers shrink).  So the algorithm involves computing the normal amounts of soil recharge, runoff, evapo-transpiration, etc for any given place in each month.  Then an indicator is defined which looks at a deficiency in water supply (rain and stored soil moisture from earlier in the year) relative to water demand (eg evapotranspiration).

However, the PDSI is not an absolute indicator.  It is assumed that the inhabitants of West Texas, say, are accustomed to less rain and soil moisture than the inhabitants of Massachusetts.  So a level of dryness that would be situation normal in West Texas will be biblical in Massachusetts because the population and the agricultural economy are not adapted to it.  The way this is actually handled in the indicator is to introduce scaling factors designed to make the worst historical drought in each place produce roughly the same indicator value.  (And the details of this calibration procedure are another place where modern versions of the PDSI have come up with more sophisticated procedures than the very rough and ready procedure that Palmer originally used).

Finally, the indicator defines a procedure for when to decide that a drought has started and to begin accumulating the scaled deficit of moisture demand over supply.

My own reaction in studying the algorithm is that it's very much a product of an earlier era - I think a modern scientist approaching the same problem would likely make much heavier use of statistical machinery and rely less on some rather crude-looking heuristics and approximations.  However, at the same time, it's abundantly clear that Palmer was deeply versed in the problem, put a great deal of work and thought into making the details work, and had a very pragmatic mind.  He keeps messing and messing with his formulae until he gets something that matches people's common sense intuitions about when it's a drought and when it's not (and in a number of places, compares his algorithm's result with what agricultural publications at the time were saying about conditions).

At the end of the day, the proof is in the pudding.  So let's turn now to some comparisons between the PDSI and reality (these are taken from the Dai, Trenberth, and Qian paper quoted above).  Firstly, while actual measurements of soil moisture are not widely available, they are available in some places, and that allows comparison between the actual measured amount of water in the soil column, and the PDSI.  That allows pictures like this one to be produced

The top picture shows actual variations in the amount of soil moisture in the top 0.9m of soil in a particular site in Illinois, and the PDSI value (for the twenty year period 1981-2001).  Clearly, there is quite a bit of correlation, but it's far from perfect.  The second plot is particularly interesting, and shows the correlation between PDSI and soil moisture content as a function of depth (y-axis), and month of the year (x-axis).  Clearly the PDSI is best as an indicator of the state of the upper couple of feet of soil in the summer, and is pretty useless for saying anything about the winter.  This is perhaps not surprising given the lack of modeling of snow, and that there probably just isn't much variability in winter soil moisture to correlate with.  Deep moisture may be correlated with PDSI with a lag, but the paper doesn't explore this.

More compelling are plots of river-flow with basin-wide PDSI for all the large rivers of the world with enough data (along with a few small rivers).  Those plots look like this:

Here the dashed lines are the basin-wide PDSI, and the straight lines are the river-flow (from stream gauges).  You can see these are, again imperfect, but pretty good.  For the most part, the PDSI seems to be capturing something important about the fluctuations in riverflow.  The Yenisey is the largest exception, and since this is a Russian river that flows into the Arctic, the lack of modeling of snow and ice in the PDSI is probably the cause of the failure (in particular, the rising streamflow and declining PDSI in the Yenisey basin suggest melting snow and permafrost).  Here are some more:

In short, the PDSI seems to be a decent approximate indicator of drought, except in high latitude (or likely high altitude) applications where snow and ice are large factors in the hydrologic cycle.

Probably the most important limitation of the PDSI for assessing the impacts of global warming is that it's  fundamentally about abnormal dryness relative to typical conditions for a particular place and season.  So, when we look at a map like this one for what climate models say about PDSI in the 2030s:

with the -4s to -8s in the red/pink/purple, it's asserting that in the 2030s we can expect conditions very dry relative to history.  What it doesn't allow us to do at all is say how France at -8 would compare to Italy at 0 (or Morroco at 0).  It's designed to measure the effects of abrupt changes on a given locality, not gradual changes.  From the perspectives of wild ecosystems, that may not matter much, since anthropogenic climate change is so abrupt compared to the normal rate of change prehistorically.  But from the perspective of human-managed landscapes, it matters a lot, since humans can certainly do a lot of adaptation over the course of thirty years (different crops, new more drought-tolerant crops, irrigation in areas where it currently isn't used, etc).

So the PDSI isn't the perfect indicator for measuring drought under global warming.  It's still pretty alarming though.

This post is part of a series on the future of drought.