I find the Failed States Index interesting, since it comes out on an annual basis (published by Foreign Policy magazine), and it is a very influential survey. The details of the methodology used to produce the index are proprietary, however. The Fund for Peace, the creator of the index, does not allow other researchers to open up their black box. Compared to other surveys (e.g. by the Brookings Institute), The Failed States Index (FSI) definitely over-states the incidence of state failure.
Obviously, it is a difficult task to rate every state in the world in terms of “stability” or what political scientists refer to as “stateness” (i.e. effective governance and internal political legitimacy). That said, I wonder if the coding software used to compile the Failed States Index is deeply flawed from an epistemological perspective (“How do they know what they know?”). As I understand the FSI, it is the result of scanning and coding of tens of thousands of documents. Presumably, most of those are secondary documents, and therefore susceptible to the propagation of bias and error from primary sources and journalists. It is fair to say that the Failed States Index is, to a great extent, a survey that reports on perceptions of states. In many cases, there is systematic bias in (outsiders’) perceptions of some states. The Fund for Peace is not a large organization and they do not have the resources to empirically monitor key patterns globally on an annual basis. That is not to say their index is without value. Only that it must be received for what it is. Clearly there is validity in some of the FSI results. But enough cases are called into question that the index as a whole suffers. Its representation of Africa as a whole is deeply problematic.
For another view of “state failure,” see this 2008 report from the Brookings Institute. Yes, the Brookings report is now a few years old, but the methodology is far different from the FSI. And it provides a far different perspective on the geographic scope of state failure.
*** Did you like what you read here? You might be interested in the new book by this blog’s author, Failed States: Realities, Risks, and Responses.