John Calipari evokes many different emotions in sports fans.
If you’re a Kentucky fan, you probably love Calipari. In six years as coach, he has won a national championship, and his 2015 team might win another with an undefeated record.
If you’re not a Kentucky fan, Calipari represents all that’s wrong with college basketball. His teams at Massachusetts and Memphis had to vacate wins during Final Four years because of NCAA rules violations. While the NCAA never found Calipari guilty of anything, it seems unlikely he knew nothing about the infractions.
Moreover, Calipari pisses off his colleagues. At a press conference, he got former Temple coach John Chaney so mad that Chaney attacked him, yelling “I’ll kill you.” Yes, this really happened; check out the video.
Numbers reveal a third side Calipari: he’s an amazing tournament coach. This article looks at how teams perform in the tournament compared with a regular season baseline. With a high degree of statistical certainty, Calipari’s teams play better in March than the regular season.
Calipari’s ability to get more out of his teams during the tournament is neither a typical part of his narrative nor the story in which I was originally interested. Michigan State coach Tom Izzo usually gets praised for his excellent coaching in March. I didn’t believe this conventional wisdom, so I dug into the numbers.
Comparing tournament performance with the regular season
To test Michigan State’s play in the tournament, I compared their margin of victory in the post season with expectations from the regular season.
For a regular season baseline, I used my college basketball team rankings at The Power Rank. Developed from my Ph.D. research in statistical physics, this algorithm takes margin of victory and adjusts for strength of schedule.
From 2002 through 2014, the higher ranked team in my pre-tournament rankings won 71.3% of games. In addition, the rankings provide a predicted margin of victory in each game. We’ll use this prediction as a baseline for tournament performance since, unlike the point spread in Vegas, it makes no preference for Michigan State or any other team in March.
For all tournament games from 2012 through 2015, I looked at Michigan State’s actual margin of victory compared with The Power Rank’s prediction. For example, in 2015, Michigan State beat Virginia by 6 points and exceeded the baseline prediction by 12.1 points. In 43 games tournament games, Michigan State has exceeded their expectation from The Power Rank by an average of 2.07 points.
Two points might not seem like a lot, but it’s a huge jump in performance. If the betting markets favor a college basketball team by 2 points, this teams wins the game 58.4% of the time, much more than the 50% for a game with a zero point spread.
Are these results statistically significant?
However, we can’t just assume that Michigan State performs better in the tournament based on this 2.07 points. There’s randomness in this estimate. We don’t know whether Michigan State performed at the same level as the regular season and got lucky by two points a game. Or Michigan State could be 4 points better than the regular season and got unlucky in the tournament.
Statistics gives us tools to account for the randomness in this estimate. A t-test, a method first developed at the Guinness Brewery, provides a probability that this estimate of 2.07 points is better than zero. This test, using this nifty calculator, gives a 92.6% confidence that Michigan State performs better in the tournament. (For those who want to check my work, the standard deviation of sample mean over 43 games was 1.40 points.)
I was wrong. The numbers suggests a high likelihood that Tom Izzo’s teams perform better in the tournament. Conventional wisdom wins this time.
The greatness of John Calipari
With the code to perform this test for Tom Izzo, I decided to repeat the test for the other Final Four coaches in 2015.
Duke coach Mike Krzyzewski has a reputation for getting the most of his players in March. However, since 2002, they have performed 0.41 points worse than The Power Rank’s expectation from the regular season.
Wisconsin coach Bo Ryan did slightly better than Krzyzewski in the tournament. However, his teams still performed 0.21 points worse than expected over 37 games.
John Calipari was a different story. During his years at Kentucky (2010 to present), his teams have performed 3.86 points better than their regular season expectation. Even with the smaller sample size than the other coaches (25 games), we can be 98.1% sure Kentucky has played better in the tournament.
Calipari also coached at Memphis before taking the Kentucky job. From 2002 through 2009, his Memphis teams exceeded their regular season expectation by 1.38 points in 20 tournament games.
It’s probably best to combine the tournament performances of Calipari’s Kentucky and Memphis teams, which gives a 2.76 point improvement over 45 games. That implies a 96.8% confidence that his teams play better in the tournament. In addition, Calipari’s tournament improvement is 35% larger than Tom Izzo’s improvement.
The following list summarizes the difference in tournament performance from the regular season since 2002.
- John Calipari: + 2.76 points per game.
- Tom Izzo: +2.04 points per game.
- Bo Ryan: -0.21 points per game.
- Mike Krzyzewski: -0.41 points per game.
For the 2015 Final Four, the main story should be John Calipari’s greatness as a tournament coach.
I’d control for age under the theory that younger teams get better over the year than older teams. Then if Calipari’s outperforms the prediction in the Tournament, then you got something. I believe KenPom has a measure of experience, though I don’t know how far back it’s available.
Thanks, Joe. Not a ton of teams that young, but definitely something to look into.
The best you can save from your data is that it confidently indicates that teams coaches by Calipari perform better in the tournament. That is not the same as saying he is a great tournament coach. It could be he’s a terrible regular season coach. Or age and experience might be the largest contributing variable to a change in performance from early to late season for college teams.
I think your first step is to control for age or take it into account but age factored by minutes played. As obviously a bench warmer’s age is going to matter very little in changing team performance over time.
Also I think you have to try and control for home vs road noting that since he’s been at Kentucky every year they are within 300 miles of Lexington they are essentially playing a home game. Other teams also travel well with fans, so in many cases some other teams might also be benefitting from that advantage. Duke for example is almost always playing road games in the tourney as they routinely face hostile crowds even in NC close to their campus.
Do some more controls. Rephrase your hypothesis/conclusions so it is actually supported by your evidence and you might have a stronger case.
Still a very interesting look and I think Cal both deserves more credit than he gets in most circles and yet still clearly is a coach who has not succeeded at the level his recruited talent would suggest over the last decade.
Problem with your analysis is you have forgotten all about esPn’s “star” system of grading recruits. Given that Calipari has probably 3 time the cumulative “stars” on his bench during the introductions, I’d say that compared to Coach Izzo, he underperforms every year he doesn’t win it all. The rest of your argument is simply statistics manipulated to fit your outcome.