Monday, November 15, 2010

Making Quant IR Credible

. Monday, November 15, 2010

Phil Arena, IR assistant professor at SUNY-Buffalo, has a new blog. He focuses on the security side of IR, but some of it crosses over. For example, this post on the difficulty of isolating causal processes from associative stats analysis. Basically the idea is this: Suppose there are two possible states of the world, one in which A causes B and one in which C causes both A and B but is not directly observable. Standard stats methods would not be able to distinguish between the two.

I think he does a pretty good job of describing the problem, so I'm not going to rehash his post. Just go read it. And I think we're seeing an increased use of Bayesian stats, instrumental variables, experimental approaches, and network analysis to try to mitigate the problem. In other words, I think things might be improving as the discipline matures. I completely disagree with this, however:

If you ask Jas Sekhon, one of the most talented methodologists we have in political science, he'll tell you that the answer for IR scholars is to give up on quantitative work altogether, learn how to do credible qualitative, and start asking more policy relevant questions.

An argument for better stats (or better theory) is not an argument for qualitative methods, which must be made on its own merit. I like qualitative methods and value their inclusion in the discipline, but it's not as if qualitative analysis is definitionally error-free, and "policy relevant" is very much in the eye of the beholder.

I also think that his suggestion that we focus more on theory -- which is unsurprising, since he does formal theoretical work -- is pablum. True pablum, but pablum nonetheless. Of course creating credible, rigorous theory is important and perhaps under-valued in IR, but the whole point of using stats is to evaluate theory. We don't know if a theory is credible or rigorous until we put it to some evidence-based test. Internal logic is important, but is not the end of the story.

Nevertheless it's a good post, and worth thinking about.

Note: Updated slightly for clarity shortly after posting.


Making Quant IR Credible




Add to Technorati Favorites