Saturday, January 2, 2010

Moneyballing Hollywood

. Saturday, January 2, 2010

Paging Michael Lewis:

Before Relativity commits to financing a particular movie — either through its slate deals with Sony and Universal or on its own — it's fed into an elaborate Monte Carlo simulation, a risk-assessment algorithm normally used to evaluate financial instruments based on the past performance of similar products. Enough variables are included in the Monte Carlo for Wilson and his team to have reached the limits of their Excel's sixty-five thousand rows of data: principal actor, director, genre, budget, release date, rating, and so on. After running the movie through ten thousand combinations of variables (in marathon overnight sessions), the computers will churn out a few hundred pages that culminate in two critical numbers: the percentage of time the movie will be profitable, and the average profit for each profitable run. The computers will also calculate the best weekend for the movie to be released, whether Russell Crowe will earn his salary or Sam Worthington will be good enough, and the box-office effect of an R rating versus PG-13. But for Kavanaugh, those are secondary considerations: Unless the movie shows the distinct probability of a return — no one at Relativity will reveal the precise green-light figure, but it's something like 70 percent — the script gets shredded. "Everything has to run on the principle of profit," Kavanaugh says. "We'll never let creative decisions rule our business decisions. If it doesn't fit the model, it doesn't get done."

This article is interesting to me for a few reasons: first, because it's the only time I've seen Monte Carlo simulations mentioned in Esquire; second, because it sounds like these guys need better software than Excel; third, because the push-back that Kavanaugh is getting is almost exactly the same as what Billy Beane got when he brought rigorous quantitative analysis to front offices of baseball teams (and still gets, despite the fact that his methods have been almost-universally adopted by other teams).

But more broadly, I'm interested to see whether "quants" still gain influence in the business world following the financial crisis. It's almost certainly true that the financial crisis could not have occurred they way it did without quantitative engineering (remember the Gaussian copula function at the root of the crisis?). And while Kavanaugh and his team constantly talk about "outliers" that their model fails to predict, as financial engineers did, it's also possible that they are simply over-confident in their model. Extreme events do sometimes occur, but that doesn't mean that every event not predicted by a model is an outlier; it could just be that the model is misspecified, i.e. has omitted a relevant variable that could have predicted that outcome.

It's also clear that these kinds of models are pretty good at some things but not others. Kavanaugh is clear about his goal to reduce uncertainty over profitability in the production of films. But he does that by minimizing risk, not by maximizing reward. As Kavanaugh says, they never would have made The Matrix. He rationalizes this by saying that they never would have made Waterworld either, but therein lies the rub: the difference between The Matrix and Waterworld isn't some intangible mystery of the market; the difference is that The Matrix is a very good movie and Waterworld is a very bad movie. Their model tries to proxy for "quality" (as measured by the market) by including variables for actor, director, genre, rating, etc., but any perusal of any page shows that these are all somewhat poor proxies. The point is that the equilibrium strategy that Kavanaugh is taking is not the only one or even maybe the best one: an informed strategy of greater risk/reward might outperform a strategy that minimizes risks.

On the other hand, it's pretty clear to me that there is no going back from the quants in most businesses, including Hollywood. If nothing else quants are able to arbitrage the system, which is what Moneyball was originally about and what Kavanaugh is doing. The question is whether quantitative models should inform practice or determine it. If all you can do is arbitrage, what happens when arbitrage opportunities dry up? Billy Beane is still trying to figure out the answer to that question. Moreover, if you're business model is based on arbitrage, as AIG's and LTCM's was, and a small-probability event (at least as measured by your quant models) hits, you're completely screwed. You can survive and profit for years using such a system, but one bad event will kill you when it might not otherwise.

In other words, I'll be interested to see where Kavanaugh is 5 or 10 years from now, and whether his methods are broadly adopted by the film industry.


Alex Parets said...

Using primarily those variables and relying on data from previous movies seems to me like it would bias Relativity's movie financing decisions to those types of movies that have only succeeded in the past. It seems like this would decrease creativity (at least the creativity that is financed) and the range of potential successful options that they would fund. They'd keep funding the same types of movies with the same actors over and over and more eclectic choices wouldn't be funded. This may explain why The Matrix wouldn't've been funded.

Echoing what you said, they're simply trying to minimize risk and play it safe which definitely means they'd have a better average and less bombs, but it also means that they'd hit less home runs on films. It seems like a mixed strategy where they use 80-90% of their financing budget on safer picks (defined by their model) and 10-20% on more eclectic choices would surely allow them to not miss out on so many big time, new idea films and maximize returns.

Moneyballing Hollywood
There was an error in this gadget




Add to Technorati Favorites