The NY Times Magazine has published a long article on Bruce Bueno de Mesquita, political scientist and paid prognosticator for the government and many corporations. He is one of the most prolific academics in political science, and is both very influential and highly divisive. It is probably not an overstatement to say that he has changed the orientation of the discipline more than any other figure in the past quarter century, and his orientation is towards rational choice modeling and game theory. In my mind and despite its flaws, his co-authored 2003 book The Logic of Political Survival will likely stand as the defining work of international relations and/or comparative political economy in the 2000s (although not everybody shares this opinion). Even more, he's the only international relations scholar to have his own special on the History Channel, titled "The Next Nostradamus". Bueno de Mesquita's out-sized reputation (and ego), along with the proprietary nature of much of his work and the large sums of money he receives for it, has generated plenty of detractors inside and outside the political science community. But he has his supporters too:
Bueno de Mesquita’s most regular client by far has been the C.I.A. He says he has performed more than 1,200 predictions for the agency, tackling questions like “How fully will France participate in the Strategic Defense Initiative?” and “What policy will Beijing adopt toward Taiwan’s role in the Asian Development Bank?” In 1987, Stanley Feder, a research political scientist for the C.I.A., published a report analyzing forecasts that Bueno de Mesquita’s firm did of political events in 27 countries; he found that the success rate of its predictions was the same as that of the C.I.A.’s own analysts, only more precise. (He “got the bull’s-eye twice as often,” Feder wrote in his report, which was declassified in 1993. No other reports have been declassified since.)
Sounds great. So where's the rub?
Donald Green, a political scientist at Yale, questions whether Bueno de Mesquita is serving the discipline well. “When I see clips of Bruce at the TED conference,” he says, referring to the annual conference promoting ideas in technology, entertainment and design, “I watch the video and I think, Wow, this is so far from the typical way in which political scientists of any stripe behave.” Some political scientists are openly dubious about the accuracy of Bueno de Mesquita’s model. Stephen Walt, a Harvard professor of international affairs, says that Bueno de Mesquita’s nonprediction work — like his theory of the “political survival” of heads of state — make him a “respected scholar, deservedly so.” It’s the predictions that Walt doesn’t trust, because Bueno de Mesquita does not publish the actual computer code of his model. (Bueno de Mesquita cannot do so because his former firm owns the actual code, but he counters that he has outlined the math behind his model in enough academic papers and books for anyone to replicate something close to his work.) While Bueno de Mesquita has published many predictions in academic journals, the vast majority of his forecasts have been done in secret for corporate or government clients, where no independent academics can verify them. “We have no idea if he’s right 9 times out of 10, or 9 times out of a hundred, or 9 times out of a thousand,” Walt says. Walt also isn’t impressed by Stanley Feder’s C.I.A. study showing Bueno de Mesquita’s 90 percent hit rate. “It’s one midlevel C.I.A. bureaucrat saying, ‘This was a useful tool,’ ” Walt says. “It’s not like he’s got Brent Scowcroft saying, ‘Back in the Bush administration, we didn’t make a decision without consulting Bueno de Mesquita.’ ” Other academics point out that rational-actor theory has come under increasing criticism in recent years, as more evidence accumulates that people make many decisions irrationally.
Academics hate it when others get a bunch of glory (and money) using proprietary data, as Bueno de Mesquita does in his private consulting. It makes it impossible to verify the legitimacy of the results. On the other hand, few argue that Bueno de Mesquita's academic work is not of quality even if they argue with his conclusions; he has been one of the most published political scientists of the past few decades, and his work has appeared in all of the top journals. His academic work has used public data, transparent sources, and has been replicated (and disputed) by many other scholars. And he's certainly not the only arrogant academic. So the animus towards Bueno de Mesquita comes almost entirely from his work outside of the academy.
Still, academics are highly skeptical of a model that can supposedly predict everything. Why? Because it can't:
[I]t’s true that there have been cases when Bueno de Mesquita’s model has gone awry. In his 1996 book, “Red Flag Over Hong Kong,” he predicted that the press in Hong Kong “will become largely a tool of the state” — a highly debatable claim today. (In 2006, Reporters Without Borders noted concerns about self-censorship but said that “journalists remain free in Hong Kong.”) In early 1993, a corporate client asked him to forecast whether the Clinton administration’s health care plan would pass, and he said it would.
What’s more, with corporate clients in particular, there’s always the potential problem of reflexivity, of the prediction itself influencing events and making it hard to evaluate the prediction’s value. Suppose a firm is told a merger will fail, for example, and abandons its merger efforts. Was the prediction accurate or a self-fulfilling prophecy?
On the other hand, Bueno de Mesquita's track record -- what we know of it, anyway -- is probably better than anyone else's.
His new book, the first targeting a general audience, comes out next month and looks poised to be the next BlackFreakoOutliers. The NYT Mag article is well worth reading in full. Below is his TED talk from this year.