Moneyballing Hollywood

Paging Michael Lewis:

Before Relativity commits to financing a particular movie — either through its slate deals with Sony and Universal or on its own — it's fed into an elaborate Monte Carlo simulation, a risk-assessment algorithm normally used to evaluate financial instruments based on the past performance of similar products. Enough variables are included in the Monte Carlo for Wilson and his team to have reached the limits of their Excel's sixty-five thousand rows of data: principal actor, director, genre, budget, release date, rating, and so on. After running the movie through ten thousand combinations of variables (in marathon overnight sessions), the computers will churn out a few hundred pages that culminate in two critical numbers: the percentage of time the movie will be profitable, and the average profit for each profitable run. The computers will also calculate the best weekend for the movie to be released, whether Russell Crowe will earn his salary or Sam Worthington will be good enough, and the box-office effect of an R rating versus PG-13. But for Kavanaugh, those are secondary considerations: Unless the movie shows the distinct probability of a return — no one at Relativity will reveal the precise green-light figure, but it's something like 70 percent — the script gets shredded. "Everything has to run on the principle of profit," Kavanaugh says. "We'll never let creative decisions rule our business decisions. If it doesn't fit the model, it doesn't get done."

This article is interesting to me for a few reasons: first, because it's the only time I've seen Monte Carlo simulations mentioned in Esquire; second, because it sounds like these guys need better software than Excel; third, because the push-back that Kavanaugh is getting is almost exactly the same as what Billy Beane got when he brought rigorous quantitative analysis to front offices of baseball teams (and still gets, despite the fact that his methods have been almost-universally adopted by other teams).

But more broadly, I'm interested to see whether "quants" still gain influence in the business world following the financial crisis. It's almost certainly true that the financial crisis could not have occurred they way it did without quantitative engineering (remember the Gaussian copula function at the root of the crisis?). And while Kavanaugh and his team constantly talk about "outliers" that their model fails to predict, as financial engineers did, it's also possible that they are simply over-confident in their model. Extreme events do sometimes occur, but that doesn't mean that every event not predicted by a model is an outlier; it could just be that the model is misspecified, i.e. has omitted a relevant variable that could have predicted that outcome.

It's also clear that these kinds of models are pretty good at some things but not others. Kavanaugh is clear about his goal to reduce uncertainty over profitability in the production of films. But he does that by minimizing risk, not by maximizing reward. As Kavanaugh says, they never would have made The Matrix. He rationalizes this by saying that they never would have made Waterworld either, but therein lies the rub: the difference between The Matrix and Waterworld isn't some intangible mystery of the market; the difference is that The Matrix is a very good movie and Waterworld is a very bad movie. Their model tries to proxy for "quality" (as measured by the market) by including variables for actor, director, genre, rating, etc., but any perusal of any www.imbd.com page shows that these are all somewhat poor proxies. The point is that the equilibrium strategy that Kavanaugh is taking is not the only one or even maybe the best one: an informed strategy of greater risk/reward might outperform a strategy that minimizes risks.

On the other hand, it's pretty clear to me that there is no going back from the quants in most businesses, including Hollywood. If nothing else quants are able to arbitrage the system, which is what Moneyball was originally about and what Kavanaugh is doing. The question is whether quantitative models should inform practice or determine it. If all you can do is arbitrage, what happens when arbitrage opportunities dry up? Billy Beane is still trying to figure out the answer to that question. Moreover, if you're business model is based on arbitrage, as AIG's and LTCM's was, and a small-probability event (at least as measured by your quant models) hits, you're completely screwed. You can survive and profit for years using such a system, but one bad event will kill you when it might not otherwise.

In other words, I'll be interested to see where Kavanaugh is 5 or 10 years from now, and whether his methods are broadly adopted by the film industry.

The Power of the Quants.

Really interesting article from Newsweek on Paul Wilmott, an Oxford-trained mathematician specializing in quantitative finance. Wilmott seeks to train newly minted physics, financial engineering and math PhD's who are out working in finance that there is a hell of a lot more to finance than simply mathematical model-building and equations. Here is an excerpt, but do check out the entire article. Worth the read.

Imagine an aeronautics engineer designing a state-of-the-art jumbo jet. In order for it to fly, the engineer has to rely on the same aerodynamics equation devised by physicists 150 years ago, which is based on Newton's second law of motion: force equals mass times acceleration. Problem is, the engineer can't reconcile his elegant design with the equation. The plane has too much mass and not enough force. But rather than tweak the design to fit the equation, imagine if the engineer does the opposite, and tweaks the equation to fit the design. The plane still looks awesome, and on paper, it flies. The engineer gets paid, the plane gets built, and soon thousands just like it are packed full of people and sent out onto runways. They fly for a while, but eventually, because of that fatal tweak, they all end up crashing.

In a way, this is what's happened in quantitative finance. The planes are the complex derivatives—like collateralized debt obligations—that now lie smoldering on the balance sheets of banks. The engineers are the "quants": those math and science Ph.D.s who flocked to Wall Street over the past decade and used mathematical models to build these new investment products. These are the people Warren Buffett was talking about when he said, "Beware of geeks bearing formulas" in his letter to shareholders this year. The quants aren't entirely to blame for the financial meltdown; there's plenty of guilt to be shared by regulators, top executives and the investors who bought the instruments the quants created. Yet while aeronautical engineers who willfully designed a faulty plane might be on trial for criminal negligence, Wall Street's math gurus are, for the most part, still employed. Strangely, the banks need quants more than ever right now. If anyone's going to figure out how to price these toxic assets, it's them. Quantitative finance isn't going away, but it is in desperate need of reform. And one man—a math geek himself—thinks he knows where to start.

This is not only true of the world of finance. This can also be extended to economics and political science. Are some rising academics simply bending equations and formulas to fit the real world, implying causality and correlation where none really exists? In the face of mounting pressures to public or perish, how much of what is put out there in our scholarly journals dishonest, mathematical model building simply for the purposes of another publication and veiled by mathematical complexity and greek symbols? 

(ht: MR)

Ah, it was the Gaussian Copula Function!

Mispriced risk.  The global economic contagion was created by mispricing (read, underestimating) risk.  And apparently this is all a Canadian-educated Chinese mathematician's fault (way to externalize blame!)

So, for me the question is how do governments create policy regimes that encourage proper risk-management?  Obviously, powerful governments have a credible commitment problem since the costs of letting a financial institution dissolve in the face of risk bets gone bad are untenable.  But, as the parable of the Gaussian Copula Function tells us, there is little incentive for financial institutions to moderate or to be cautious.  So, our outcome seems doomed to be Pareto sub-optimal.

Even more concerning is industry and governments proclivity toward over-compensation.  When this financial mess bottoms out and we recommence our dogged climb toward ever-increasing prosperity, there will probably be a heck of a lot of regulation concerning the ways in which CMOs and credit default swaps can be created, rated, bought, and sold.  But, the underlying cause of financial blow up was not crazy mortgage-backed derivatives per se, rather risk mis-management stemmed from a deeply human tendency to discount "outliers" and treat events that occur under a certain threshold of probability as practically impossible.  And, until risk calculations take correlation seriously, financial firms in their legal obligation to produce the greatest returns for shareholders will continue to find ways to delude themselves and their clients into believing that you can make lots of money without taking on any risk.