THE MYTH OF MONEYBALL
In 2003, writer Michael Lewis published a seminal book about baseball called ‘Moneyball’.
It told the story of Billy Beane, the General Manager of major league baseball team the Oakland A’s, and how he turned them into champions by using data rather than old-fashioned baseball thinking.
The book was so cool it got made into a film starring Brad Pitt (doesn’t get much cooler than that) and was so popular it spawned a whole industry – Sports Analytics.
All over the world, smart young numbers geeks read Moneyball and thought ‘I could do that for football (or basketball or American Football or rugby or golf or cycling or hockey.….)’.
A community of these like-minded people formed, and now they meet every year at the MIT Sloan Analytics Conference on the East Coast of America, and at events like the OPTAPro Analytics Forum.
There are myriad sports analytics blogs, some books and a few consulting services.
Sports clubs globally have invested heavily in analytics in the hope of finding themselves a slice of the ‘Moneyball magic’.
But what if Moneyball is a myth?
“Analytics don’t work at all. It’s just some crap that people who were really smart made up to try to get in the game because they had no talent.” Charles Barkley, Hall of Fame NBA basketball player.
What if Charles is right, and Analytics doesn’t work at all?
Let’s have a look, starting with the fundamentals…
WHAT IS ANALYTICS?
What is Analytics? – It’s the scientific process of transforming data into insight.
Why is it attractive? – It’s cool and interesting and clever (although only if you like that sort of thing).
What is it useful for? – It is a tool that can be used to make more efficient decisions.
So Barkley is basically wrong – saying that analytics doesn’t ‘work’ is like saying a saw, a screwdriver or a hammer doesn’t work. It is just a tool. No tool works independently of human control. Whether or not it helps you in your task depends on how you use it.
Think of Sports Analytics as a powersaw. It’s a tool that can help you do things (like cutting a big plank of wood, or identifying undervalued transfer targets) much more efficiently than by trying to do it without a tool. But in the wrong hands or used in the wrong way it has the potential to make things a whole lot worse rather than better. If you don’t know what you’re doing, you could lose some fingers.
Where Barkley is right is when people talk about analytics being ‘revolutionary’, or a ‘game-changer’. It’s not. It’s just a tool. No matter how cool, interesting or clever it is – it’s still just a tool that can help decision makers make more efficient decisions. It won’t ever have a monopoly on that. And it certainly can’t be used to prove that everything we know about a sport is wrong.
Football is subject to lots of quantitative analysis these days. But a game of football is only ever twenty-two sweaty young men running around a field kicking a ball and each other, no matter how many data points you capture while they do it.
THE OAKLAND A’s
Moneyball is about the Oakland A’s baseball team. They aren’t one of the richer teams in MLB, so Billy Beane used some of the analytic principles proposed by the likes of Bill James to find inefficiencies in the way baseball teams were managed, and did things a little differently. The book and the film portrayed this as a storming success, turning the no-hopers into title contenders.
They say never let the facts get in the way of a good story, but the reality is that the A’s have only won one single playoff series in all of Billy Beane’s near two decades as GM.
They’ve qualified for the playoffs 8 times in Beane’s 18 years at the franchise – not bad, but not exactly great either. This season they have the worst record of any of the 15 teams in their American League (43.7% wins). In its history the Oakland A’s have won 48.7% of their games. In Billy Beane’s time they have won 53.9%
Billy Beane and Sports Analytics has NOT turned the A’s into a champion baseball team. At most we can say they have been a slightly better team than they ‘should’ have been based on their budget.
This is an achievement not to be sniffed at. In pro sports it’s always impressive to consistently out-perform your financial means. But it doesn’t qualify the A’s as a miracle, or Beane as a genius. He’s probably just a smart, slightly above average GM who has managed to make some smart decisions using analytics, making his team a little bit more efficient than they should be. He got played on film by Brad Pitt though, so he’s never going to be short of work.
In other words, if Billy Beane was a football manager he would be Sam Allardyce or Tony Pulis. Not sure the DVD sales for the movie would have gone quite as well with either of those pair on the sleeve instead of Brad!
THE COIN FLIP PRINCIPLE
The difference that can be made by making things just a little more efficient shouldn’t be under-estimated though. It works on the ‘coin flip principle’.
Baseball conforms to the principle. Winning or losing a baseball game is essentially a coin flip, where the coin is weighted slightly in favour of the better (and home) team. The San Francisco Giants won last year’s World Series, but they only won 55% of all their matches in the season. The best record in the whole of the MLB regular season was the Angels with 60% wins.
In coin-flip situations the best teams don’t win all their games, or anywhere near. Their superiority is expressed in small increases from the 50% win rate they would get if they were ‘average’. Even if you’re exceptional, you’ll still only win about 60%. You can weight the coin, but you can’t make it fall on the same side EVERY time.
Lots of other things conform to the coin-flip principle, including betting on Asian handicap markets in football. If you just selected Asian Handicap and Asian Over-Under bets randomly while wearing a blindfold, over the course of a season you’d still win with about 50% because the outcome is a coin-flip (though you’d lose money t bookmaker because the average price you would get is a little worse than Evens). The very best Asian market bettors in the world win about 60% of the time. That small-sounding 10% edge over the ‘average’ bettors though can be translated into enormous profits when it’s exploited over and over again, with large stakes.
A football club’s dealings in the transfer market conform to the coin-flip principle. The rule of thumb is that 50% of a club’s senior signing will be a success (as might be defined by the player becoming a first team regular). The % varies a little according to how much is spent on a transfer fee, but not by much. It’s essentially a coin flip. The very best transfer market operators can hope to do better than average, and get something like 60% success. That 10% difference is what analytics can help you achieve, although there are certainly other ways of doing it too.
Analytics is a tool that is good (in the right hands) for jobs like deciding transfer targets. It can make a club a bit more efficient. What it can’t do is make the coin flip into a 100% sure thing. Such a thing is a literally impossibility, because a whole host of outside factors come into play to decide whether a signing will be a success.
No analytic model could have foreseen that Angel Di Maria’s house would be burgled and he would lose form and then want to move away from Manchester as a result. The universe is subject to randomness. Stuff just happens, and there’s no way to predict it. The best you can do is be efficient in the face of the inevitable random nature of sport/chance/life.
So Barkley is right to say analytics ‘doesn’t work’ if by ‘work’ you mean ‘guarantee 100% success’. But as long as you view analytics in the right way, as a tool to help with efficient decision making, then it absolutely can and does work. So long as the tool is skillfully used.
The real hero of Moneyball is Michael Lewis. He’s a genuinely terrific writer. In less skilled hands Moneyball would have barely been a story at all – “baseball GM uses some data based theories developed by people like Bill James, and makes an average team slightly better than average” – into a worldwide phenomenon.
He’s a great writer not just because Moneyball and Liar’s Poker are great books, but because he is also the guy who wrote the article about Shane Battier – the ‘No Stat All-Star’. Good analytics is about a lot more than finding guys with ‘good stats’. Talent doesn’t exist in a vacuum. Nor is it easy to identify talent with numbers alone.
ANALYTICS IN FOOTBALL
In the realm of football betting, when OPTA started doing detailed match stats for Premier League games, loads of bettors shouted ‘Eureka!’ With all these amazing stats to work with, giving them unprecedented insight into the underlying numbers that the teams’ previous games had produced, they thought they were bound to clean up betting on football now.
But knowing lots about football doesn’t guarantee you any success betting on it. Just like knowing lots of football stats doesn’t guarantee you will run your football club more efficiently.
Everything in the universe (and certainly a game of football) is subject to randomness, so no-one can know for sure what is going to happen in the future, no matter how much they know about the past.
At OddsModel we use analytics to better understand sports like football, and Fantasy Sports games. The quantitative models we build help our betting clients get to around a 60% betting success rate.
We build quantitative models to scientifically measure how good teams and players are, and pricing engines to work out how likely various outcomes are to occur. These are intelligent guesses, but they are still guesses. We manage to weight the coin in our clients’ favour a little bit. The analytics element is only ever a part of the efficient decision making process. Data alone without regard to context is never enough – in fact is it downright dangerous.
Good analytics is excellent at exposing inefficiencies. Inefficiencies exist where things are done a certain way because ‘that’s how we’ve always done them’ or where behaviour is shaped by lazy, superstitious or out-dated thinking. As an objective scientific pursuit, analytics can cut through to the heart of something and expose it as it really is, rather than how it has always been assumed to be. Exploiting inefficiences in markets and human behaviour lies at the heart of what many of the world’s most successful people do.
Football clubs investing in analytics will find this. There is no magic formula for success waiting to be discovered in their databases of billions of data points. But there will be inefficiencies in the way they currently operate, which they can improve if they can recognise them and find better ways to do things.
Analytics can never offer a shortcut to success. Although of course that doesn’t stop people trying to sell the idea that it can.
THE HYPE CYCLE
The truth is that Sports Analytics conforms to the Hype Cycle of Innovation.
OPTA stats and Moneyball are the ‘First-generation products’ in the Technology Trigger section.
Football betting is well ahead of ‘football’ on the curve. Betting syndicates and those companies who supply them with data and intelligence are currently travelling along the Slope of Enlightenment. Analytics in football is just starting on the downward curve at the end of the Peak of Inflated Expectations. (Admittedly some of the Liverpool fans who saw Damian Comolli spend £55m on Andy Carroll and Stuart Downing because of their impressive ‘last third possession regain’ stats will have been wallowing in the Trough of Disillusionment for a while now). Moneyball’s honeymoon period is over and the negative press articles are starting to appear.
“Some of the smartest guys in football don’t work in football itself, they work in the betting industry.” Rasmus Ankersen, Chairman of Danish champions FC Midtjylland.
One football club which is probably well ahead of most others in their use of Analytics is Brentford. They are owned by Matt Benham who owns the Smartodds betting syndicate and it’s in-house Quant team. As an analytics ‘brains-trust’ they have Phil Giles (former head of the quant team at Smartodds), Ted Knutson (former pro punter and Pinnacle trader) and Rasmus Ankersen (chairman of Danish champions Midtjylland). Their club has the lowest revenue in the Championship, but it’s a fair bet that the ‘brains-trust’ will make them efficient enough to get close to promotion to the Premier League over the next few seasons..
Pro betting people have lived through the Peak, the Trough and ridden the Slope towards the Plateau. They will have a better idea of what works and what doesn’t than most ‘proper’ football people trying to use football analytics to get an edge.
THE BETTING LABORATORY
Analytics is a science, and science is done best when it invokes the ‘scientific method’. The method involves proposing theories, and testing those theories through analysis and experimentation. Then drawing conclusions and making proposals, and submitting these for peer review.
Betting provides brilliant laboratory conditions for doing the science of analytics. Bettors come up with theories and models which they use to bet. If they win (over a long term, or against a bank of historical price data) they know they were right. If they lose, they were wrong – back to the drawing board. This is is an incredibly effective feedback loop.
No such feedback loop exists in football. A club can come up with a theory based on analytics and implement it. But they have no way to know if it was actually successful, because it is always possible that they would have been more successful without it. There is no way of knowing for sure. And short-term success can sucker the foolish into believing in a direct relationship between cause and effect. Correlation does not imply causation.
A club’s transfer dealings might have been better if they hadn’t used stats to find signings. Their players might have played better without being shown all their performance numbers from training. They might have prevented more injuries by relying on the judgement of a physio with an expert eye….
Analytics will be very useful to football clubs. Eventually. Those on the Slope or the Plateau will get a decent competitive edge for a while until the others catch up. It will weight the coin in their favour in a lot of the things that they do. But it is not the revolutionary game-changer that the Peak of Inflated Expectations-mongers prophecised it would be.
Analytics is a tool that can help to make efficient decisions. Nothing else. Beyond the hype, that is real story of Moneyball.