Saturday, January 23, 2010

Trends in Failed State Count

The chart above shows the number of countries on Alert status by year, according to the Fund For Peace/Foreign Policy Magazine Failed States Index. They've only been doing this index since 2005, so it's probably a bit too soon to say whether the methodology has fully settled down and how much natural variability there is year-to-year.

Still, the 2006-2009 increasing trend is certainly eye-catching.

However, looking at the data a different way suggests caution in over-interpreting that trend.

The index is based on giving scores on a scale of 0-10 for 12 different socio-economic indicators of badness. Then all the scores are added together.  Countries with a score over 90 are on "Alert" status, whereas countries over 60 are "Warning" status. The variables look generally reasonable to me (things like "Massive Movement of Refugees or Internally Displaced Persons" are clearly a bad sign), but most of them involve at least a certain amount of subjective judgement. For example, as a US resident, I'm certainly willing to agree that "Rise of Factionalized Elites" is not a good thing in a country, and that the US deserves it's high-for-a-developed-country score of 4.0 on that. OTOH, it's clearly not the kind of thing that can be measured with too much exactitude.

Just to give you a little more feeling for it, without you having to click the link above and go off and read it all, in 2009, the top ten most failed countries (with score) were:

5Democratic Republic of the Congo108.7
8Central African Republic105.4

Worth noting that the al-Sharistani plan is to occur in #6 on the global list of failed states...

The top ten most stable countries, according to this list, were:

7New Zealand23.3

Here's the exact list of variables:
  • I-1. Mounting Demographic Pressures
  • I-2. Massive Movement of Refugees or Internally Displaced Persons creating Complex Humanitarian Emergencies
  • I-3. Legacy of Vengeance-Seeking Group Grievance or Group Paranoia
  • I-4. Chronic and Sustained Human Flight
  • I-5. Uneven Economic Development along Group Lines
  • I-6. Sharp and/or Severe Economic Decline
  • I-7. Criminalization and/or Delegitimization of the State
  • I-8. Progressive Deterioration of Public Services
  • I-9. Suspension or Arbitrary Application of the Rule of Law and Widespread Violation of Human Rights
  • I-10. Security Apparatus Operates as a "State Within a State"
  • I-11. Rise of Factionalized Elites
  • I-12. Intervention of Other States or External Political Actors
And, for example, "Rise of Factionalized Elites" is defined as:
  • Fragmentation of ruling elites and state institutions along group lines
  • Use of nationalistic political rhetoric by ruling elites, often in terms of communal irredentism, (e.g., a "greater Serbia") or of communal solidarity (e.g., "ethnic cleansing" or "defending the faith"
But as far as I can tell, assigning the actual numerical value for one of these twelve measures to a given country is a purely subjective choice by the judges. I'm not criticizing that - it's not obvious to me what else one can do - but one thing that would help a lot in scientific assessment of this kind of thing is to use multiple judges and then do statistical analysis of how much consistency there is between different judges, and whether they tend to make stable judgements over time. As far as I can tell from poking around the website, that isn't done here. So the index really represents the subjective judgement of the staff of the Fund for Peace. That's probably a lot better than useless, but does present a lot of potential for drift in standards over time, uneven application of standards between countries, etc.

For example, I wouldn't question their judgement that Somalia is a lot more failed than Norway! But if Somalia and Zimbabe change places at the top of the list, that's unlikely to be a meaningful event given the tiny difference in scores. It would be very helpful to have a more solid methodology so it would be possible to put an error bar on the index, and then assess whether or not changes were actually likely to be meaningful.

So then, when we look at the distribution of scores, here shown as the cumulative distribution:

I think it raises questions about the meaning of trends in the count of states in the "Alert" status (which on this graph is the y-intercept of where the data curve crosses from the orange zone into the red zone). For example, the 2009 curve is a little to the right of the others throughout the whole curve. Did the stability of Norway and Finland really degrade a little bit in a manner similar to that of Somalia and Zimbabwe at the opposite end of the curve? Or is it more likely that the scoring standards changed a little bit in some way?

I think I'd want to see significantly larger movement in the curve before drawing the conclusion that the world was really trending towards less stable at this time.


Stuart Staniford said...

Incidentally, what is it with Scandinavia always being on top of these kinds of lists? My speculation is that all the disruptive, aggressive, risk-taking elements left the gene pool on Viking longships back in the ninth century, and ever since then, the remaining peace-loving, harmonious, co-operative genes have been sitting in a circle singing Kumbaya (in Swedish/Norwegian/etc of course). This is sort of the reverse of the American Mania thesis for why the US is so individualistic, aggressive, and conservative.

Any of my Scandinavian readers care to correct me?

Anonymous said...

but one thing that would help a lot in scientific assessment of this kind of thing is to use multiple judges and then do statistical analysis of how much consistency there is between different judges, and whether they tend to make stable judgements over time. As far as I can tell from poking around the website, that isn't done here. So the index really represents the subjective judgement of the staff of the Fund for Peace.

Seems like using a wide variety of "qualified" judges, tossing outliers and then simply using some statistical mean/median/weighted central tendency might be easy enough (assuming there is a willingness of the Fund folks to relinquish their direct control/impact on the data).

On 2009 being a tad to the right, this might or might not be reflective of the global financial situation, specifically with regard to sovereign debt (even more explicitly to external sovereign debt). I have no way of knowing of course, and the inclusion of Ireland in the stable 10 makes me wonder whether economic/fiscal responsibility has any impact at all.


Greg said...

Stuart, re the stability of the Scandinavian states.

I doubt that any single-factor explanation will be right. The causes lie in the intersection of geography (driving co-operation), culture, and history (which lead to small, homogeneous, careful populations).

Certainly the Swedes were successful aggressors quite some time after the Viking period ended.

But hey, I'm not Scandinavian. What do I know? ;-)

Stuart Staniford said...

Greg - I take your point. It would seem that fighting four major wars in the seventeenth century and building an empire would suggest my speculation might be a tad oversimplified. There was a day last week were about half my new visitors were coming in from some site in Sweden, so I'm still hoping a few of them will enlighten me.

Unknown said...

Well I can toss out my own two cents here. My parents had what in Minnesota is called a mixed-marriage - my father's ancestors all came from Sweden, and my mother's ancestors (including my grandmother) all came from Norway.

I guess the thing that strikes me the most is that they are much less individualistic in Scandinavia - they are more communal than anything else. I don't think it has much to do with who emigrated and who didn't - it was mainly Malthusian pressures that caused many to emigrate to America (inability to grow enough food on small family farms - esp before mechanization). In my family tree, there were oftentimes lots of kids, and it seemed like they were sending some of them to America as a way of relieving the pressures. For example, in the 1970's one of my cousins wanted to travel to Norway to visit relatives, and my grandmother was worried that they would have trouble finding enough food (there were no problems of course).

Norwegians pay taxes at levels that the rest of us could never contemplate. But they get a lot in return for that - things like health care, child care, and retirement are among the benefits. I remember a Norwegian cousin commenting about the price of petrol over there - they produce the stuff, and yet they haven't gone the route of subsidized fuel that many other OPEC nations have - instead their fuel prices are heavily taxed at levels that are comparable to the rest of Europe.

I don't recall ever hearing any concerns about "lazy" people ripping off the system. They seem to value highly having a safety net for many things.

Part of the fuel tax is to discourage waste. I get the sense in talking to people that they are well aware that the oil they produce won't last forever, and that they don't want to encourage wasteful use. Come to think of it, ostentatious displays of wealth aren't don't really seem to be socially acceptable over there the same way they are here. There is no doubt that there are wealthy people there, but you just don't see the garish displays that we have in the U.S.

After WWII, Scandinavia was fairly poor as they were recovering from the occupation. My mother recalls sending packages with clothing and food to her Norwegian cousins in the years after the war. Now that I think about it, this too may have cemented the notion in my grandmother's mind that there might still be food shortages over there.

Norway of course struck riches with offshore oil, and there is no doubt that that changed things considerably. I don't know the degree to which the oil wealth was used to subsidize the social safety net. But Sweden has no real oil wealth, and to my eyes they seem just as prosperous as the Norwegians.

I don't have any Danish cousins, so I really don't have any special observations about that country.