I read Michael Lewis’s Moneyball over the summer (you’ve probably seen the Brad Pitt adaptation). It’s a great account of how stripping baseball down to the stats allowed a small team to compete against teams with much larger budgets. What is particularly intriguing is how this multi-million dollar industry was basically doing it all wrong. Mythology, tradition, inherited wisdom created a culture where certain attributes were overvalued, and others undervalued. Players who were invaluable to a team when you looked at their stats were passed over by every single club, because their shape was wrong, or they didn’t look right when they swung a bat.
It’s hard not to read it and draw some analogies with education, and in particular the learning analytics approach. I imagine a copy of Moneyball sits on every analytics nerd’s bookshelf. There are undoubtedly parallels that can be drawn, but equally interesting is why the Moneyball approach doesn’t work in education.
Let’s consider some of those similarities first. Education is rather shrouded in mystery, folklore and received wisdom. We don’t know what works, but we know what’s good when we see it. It is an industry with a lot of money involved in it and like baseball people care passionately about it. It is also often resistant to change. To the analytical mindset the only outcome worth considering is scores. And in improving scores, I will bet there is as much in education that is irrelevant as there is in Lewis’s account of baseball. Teachers are like the wizened old scouts telling the Harvard whizkid that will never fly, and education just isn’t done like that.
There is something undeniably romantic about this vision of the outsider coming in with their new method and revealing all the wastage, all the misinformation that people have been operating with for centuries. And, I genuinely believe analytics will reveal some surprising and unsettling findings for educators, and that long-cherished beliefs about what’s important simply won’t hold up against the data.
But it’s also worth considering why education isn’t like baseball. Firstly, baseball, for all it’s romanticism and mythology, is much simpler. There are very simple, observable metrics – games won, runs scored. You can add in more, but really that’s all you need to work against. This is not the case in education, although the increased obsession with scores attempts to make it so. There are a lot of other things you’re doing in education beyond those metrics – getting students to become critical thinkers, to develop skills in groupwork, communication, reflection, etc.
The reason it isn’t the case in education brings me onto the second major difference: Baseball is ruthless. The system doesn’t need to care if a promising player doesn’t make it, they can trade for someone with better stats. It can sacrifice all to achieving those metrics (and because baseball players are paid good money, this isn’t such an ethical dilemma). This is not the case in education. While some of the prestigious universities can keep up their status by ensuring only the best enter and stay, the system as a whole wants people to progress through, even if their ‘stats’ aren’t great. For the individual, for society, it’s better to have people coming through even if in moneyball terms you’d cut them.
I blog this partly to remind myself – sometimes an analogy is powerful and we tend to over-apply it. As with the disruption (klaxon) of the record industry, people have seen education as being exactly the same. It is important to see similarities, but also to recognise key differences. Anyway, go and read Moneyball if you have the time, it’s good fun.
Nate Silver’s potted history of this in “The Signal and the Noise” is worth a read, as I am sure you have already seen. It turns out that it isn’t quite as good as has been made out.
Thanks David, haven’t seen that, I can’t say I’m surprised – there is always the imposition of a narrative and ignoring others.
One thing that makes a good movie and good book is over simplification so folks “get” the story quickly. The real problem for baseball is that to succeed is a combination of method, luck, and instinct. The same is true in education. But the stories are so much simpler if we reduce it to “method is the cause for success”. Thankfully real teachers know better and take all new “breakthrough discoveries” with a large grain of salt.
Hi Chuck, yes you’re absolutely right, there is definitely a journalistic interpretation imposing a narrative on that book. I haven’t done any research into how realistic it is, I confess. Nevertheless the general story of the As competing (and continuing to compete) has some power, and if overplayed, has some relation to their scouting approach. What you say is also interesting in the stories we construct around ed tech to ourselves.
My new favourite introductory stats book – Curve Ball by Jim Albert and Jay Bennett – introduces elementary statistics ideals in the context of baseball. It compares the traditional “sports stats” approach of generating summary statistics and looking for records with a statistician’s take on how to look more closely at the numbers to see how they compare with numbers generated by chance.
In baseball, it seems chance plays a large part; the book also shows how some favoured rules of thumb about how to rate players or teams aren’t really predictive at all.
So I wonder – what is the role of chance in this or that learning/education related metric? What useful distinctions can be made between dashboard edustats (sports stats equivalent) and more statistically robust findings (which may not have the same easily grasped, natural interpretation that many of the simpler rules of thumb do, even if they are predictively worthless).
For me, part of the power of the more informal sports stats approach is that they can lead you in to a story and get you looking more closely about how this or that result or series of events came to be, or how it provided a context within which other things were happening. That is, it can help you get your eye in.
The other reason I like them is that doing real stats properly is too dependent on distribution arcana for me;-)
Thanks Tony, yes, I should have mentioned the significance of chance, as that is a big factor. The role of chance in learning is interesting. Thanks for the book ref.
Not to mention baseball players get paid to be there 😉
Now who in education has been throwing spitballs?
🙂 yes imagine the MOOC completion rates if we paid learners $1M a year to study