Trends in Failed State Count

The chart above shows the number of countries on Alert status by year, according to the Fund For Peace/Foreign Policy Magazine Failed States Index. They've only been doing this index since 2005, so it's probably a bit too soon to say whether the methodology has fully settled down and how much natural variability there is year-to-year.

Still, the 2006-2009 increasing trend is certainly eye-catching.

However, looking at the data a different way suggests caution in over-interpreting that trend.

The index is based on giving scores on a scale of 0-10 for 12 different socio-economic indicators of badness. Then all the scores are added together.  Countries with a score over 90 are on "Alert" status, whereas countries over 60 are "Warning" status. The variables look generally reasonable to me (things like "Massive Movement of Refugees or Internally Displaced Persons" are clearly a bad sign), but most of them involve at least a certain amount of subjective judgement. For example, as a US resident, I'm certainly willing to agree that "Rise of Factionalized Elites" is not a good thing in a country, and that the US deserves it's high-for-a-developed-country score of 4.0 on that. OTOH, it's clearly not the kind of thing that can be measured with too much exactitude.

Just to give you a little more feeling for it, without you having to click the link above and go off and read it all, in 2009, the top ten most failed countries (with score) were:

Rank	Country	Score
1	Somalia	114.7
2	Zimbabwe	114
3	Sudan	112.4
4	Chad	112.2
5	Democratic Republic of the Congo	108.7
6	Iraq	108.6
7	Afghanistan	108.2
8	Central African Republic	105.4
9	Guinea	104.6
10	Pakistan	104.1

Worth noting that the al-Sharistani plan is to occur in #6 on the global list of failed states...

The top ten most stable countries, according to this list, were:

Rank	Country	Score
1	Norway	18.3
2	Finland	19.2
3	Sweden	20.6
4	Switzerland	21.2
5	Ireland	21.6
6	Denmark	23.2
7	New Zealand	23.3
8	Australia	25.9
9	Netherlands	27
10	Austria	27.6

Here's the exact list of variables:

• I-1. Mounting Demographic Pressures
  
• I-2. Massive Movement of Refugees or Internally Displaced Persons creating Complex Humanitarian Emergencies
  
• I-3. Legacy of Vengeance-Seeking Group Grievance or Group Paranoia
  
• I-4. Chronic and Sustained Human Flight
  
• I-5. Uneven Economic Development along Group Lines
  
• I-6. Sharp and/or Severe Economic Decline
  
• I-7. Criminalization and/or Delegitimization of the State
  
• I-8. Progressive Deterioration of Public Services
  
• I-9. Suspension or Arbitrary Application of the Rule of Law and Widespread Violation of Human Rights
  
• I-10. Security Apparatus Operates as a "State Within a State"
  
• I-11. Rise of Factionalized Elites
  
• I-12. Intervention of Other States or External Political Actors
  

And, for example, "Rise of Factionalized Elites" is defined as:

• Fragmentation of ruling elites and state institutions along group lines
• Use of nationalistic political rhetoric by ruling elites, often in terms of communal irredentism, (e.g., a "greater Serbia") or of communal solidarity (e.g., "ethnic cleansing" or "defending the faith"

But as far as I can tell, assigning the actual numerical value for one of these twelve measures to a given country is a purely subjective choice by the judges. I'm not criticizing that - it's not obvious to me what else one can do - but one thing that would help a lot in scientific assessment of this kind of thing is to use multiple judges and then do statistical analysis of how much consistency there is between different judges, and whether they tend to make stable judgements over time. As far as I can tell from poking around the website, that isn't done here. So the index really represents the subjective judgement of the staff of the Fund for Peace. That's probably a lot better than useless, but does present a lot of potential for drift in standards over time, uneven application of standards between countries, etc.

For example, I wouldn't question their judgement that Somalia is a lot more failed than Norway! But if Somalia and Zimbabe change places at the top of the list, that's unlikely to be a meaningful event given the tiny difference in scores. It would be very helpful to have a more solid methodology so it would be possible to put an error bar on the index, and then assess whether or not changes were actually likely to be meaningful.

So then, when we look at the distribution of scores, here shown as the cumulative distribution:

I think it raises questions about the meaning of trends in the count of states in the "Alert" status (which on this graph is the y-intercept of where the data curve crosses from the orange zone into the red zone). For example, the 2009 curve is a little to the right of the others throughout the whole curve. Did the stability of Norway and Finland really degrade a little bit in a manner similar to that of Somalia and Zimbabwe at the opposite end of the curve? Or is it more likely that the scoring standards changed a little bit in some way?

I think I'd want to see significantly larger movement in the curve before drawing the conclusion that the world was really trending towards less stable at this time.