Would a Big 12 championship game be a mistake? — Analytics on College Football Playoff odds


The Big 12 hired an analytics firm to advise them on the best way of making the College Football Playoff. After crunching the numbers, the firm suggested expanding to 12 teams, playing 8 instead of 9 conference games and having a championship game.

This idea of holding a championship game interests me the most. Navigate, the research firm, says this change alone would increase the Big 12’s playoff odds by 5 percent. (All three suggestions add up to an 10-15 percent increase.)

However, it doesn’t make sense that adding a championship game will increase a conference’s chance to make the playoffs.

Think back to 2015. Oklahoma finishes their regular season at 11-1 and 3rd in the committee rankings. Since the Big 12 didn’t have a championship game, the Sooners sat at home during the last week of the season. They couldn’t lose and fall in the rankings.

Here, we’ll use simulations to determine the effect of a championship game on a conference’s playoff chances. The results impact not only Big 12 teams but also Notre Dame.

The numbers suggest that the committee must place extremely high value on winning a championship game, not just being a conference champion, to see any kind of increase in playoff odds. Let me explain.

Analytics on conference championship games

br_playoff_oddsIn 2014, Bleacher Report asked me to calculate the chance that each team would make the College Football Playoff. I wrote about these simulation results in their weekly playoff odds report for the past two seasons.

I’m happy with the accuracy of these projections. The numbers didn’t like undefeated Utah in early in 2015, and they subsequently lost 3 games. I also predicted a difficult path for top ranked Mississippi State in 2014, a team that failed to make the playoff. You can make your own judgement based on the archive of predictions.

To look at conference championship games, I use the same methods to simulate the past three seasons retrospectively. Let’s look at the two key components to this calculation.

  • College football team rankings. For each season, I use season ending rankings to calculate a win probability for each game. This uses the best estimate of team strength to retrospectively look at the season.
  • Season long simulations. The simulation marches through each week of the season and picks winners at random according to the assigned win probability.

Each week, teams that lose drop in the committee rankings. Teams that win usually hold their place in the rankings but occasionally jump over teams ahead of them.

The simulation also determines the winners of divisions to hold conference championship games the last week of the season. The simulation accounts for conference champions before the final rankings. If a top four team is not a conference champion, they have a chance of falling out of a playoff spot (25% with one loss, 50% with two losses, 75% with three losses).

These assumptions on conference champions have an enormous impact on playoff probability.

Does a championship game give the Big 12 an edge?

Suppose the Big 12 added a conference championship game with its current 10 team structure. The simulation finds the top two teams by conference record and has them play in a championship game.

Over the past three seasons, the Big 12’s playoff chances drops an average of 17%. This differs from the Navigate Research results that show a 5% increase.

My story almost ended here. The simulations made the clear statement that a championship game would hurt the Big 12. Their top team can’t drop without a loss, so don’t hold a championship game.

And I probably should have stopped here. “Big 12 analytics firm makes stupid conclusions” makes a pretty good headline. But I wanted to see how far I could push these results. What if the simulation gave more credit to teams that played in a title game?

How will the committee view a conference championship?

Let’s push the boundaries of how the committee gives credit to a conference championship.

  • 1. Dropping teams without a conference championship. Now, a one loss team has a 50% to drop out of a playoff spot, and all two loss teams get dropped.
  • 2. Moving up for winners of conference championship games. A team that wins a conference title game has a 75% chance to move ahead of the team in front of them. This places a high value on “one last chance to impress the committee,” an opportunity Big 12 teams and Notre Dame do not have right now.

With only the first criteria of a higher drop rate for non-champions, the Big 12’s playoff chances decrease an average of 9.2% with a championship game. The drop is smaller than in my original model (17%) but still significant.

With both criteria, the Big 12’s playoff chances increase by 5.5% with a championship game. If you assume the committee will give a bump to teams that win a conference championship game, then this game helps the odds. This 5.5% increase is about the same result obtained by Navigate Research.

What about expansion to a 12 team league?

Navigate Research also suggested expanding to a 12 team league. It’s impossible to know this might impact the Big 12 without knowing which teams would join the conference.

However, a simple trick lets us study how a championship game impacts a larger conference. I can eliminate the championship game from each the other Power 5 conferences and study how this changes their playoff odds.

First, consider the strongest assumptions on a conference championship that include both criteria of the previous section. Then a conference championship gives these conferences a 1.2% advantage to make the playoff, a smaller increase than the Big 12 increase of 5.5%.

In my study over three seasons, I get three probabilities for the Big 12 and 12 for the other Power 5 conferences with more teams. This small sample size most likely explains the differences in the playoff odds between these two sets. This also suggests that the estimate of a 5.5% increase for the Big 12 is probably high.

What does analytics say about expansion?

My simulations confirm that a Big 12 championship game would increase their odds of making the College Football Playoff. However, there are numerous reasons to doubt whether this increase actually exists.

First, the simulation assumes that the committee will give a strong preference to teams that win a conference championship game. This includes the dropping of teams without a conference title and the bumping of teams that win a championship game, an opportunity that doesn’t currently exist for Big 12 teams or Notre Dame.

The results are extremely sensitive to the parameters. For example, the simulation assumes a 75% chance that a team that wins a conference title game jumps over a team without such a win. This leads to a 5.5% edge for making the Playoff if the Big 12 holds a title game.

If the simulation only gives a 50% chance to make this jump, then a championship game hurts the Big 12. Remember, this extra game exposes a team to a possible loss. With these parameters, the Big 12 playoff chances decrease by 2.8% with a championship game.

What’s the honest truth on analytics and a Big 12 championship game? We don’t know. It’s impossible for me to assign pinpoint values to the parameters of my simulations, even if I had ten years of data on the selection committee.

The committee does seem intent on giving winners of a championship game every advantage. In 2014, a flawed Florida State team jumped TCU after winning the ACC championship game. In 2015, Michigan State and Stanford both moved ahead of teams without championship game wins (Oklahoma, Ohio State respectively).

But no one knows if the committee will keep placing such a high value on conference titles. And what about top teams that lose their conference championship game? If top ranked Clemson lost to North Carolina in the 2015 ACC title game, would they have dropped from the top 4?

What do you think? Do my methods give too much credit to conference champions? Let me know in the comments.

How does Steph Curry’s injury impact the Warriors series win probability?

Steph Curry sprained his knee and will miss the next two weeks of the playoffs.

How will this impact the Warriors, the clear championship favorite with Curry? We can estimate his impact by looking at the closing spread against Houston with and without him.

For example, the markets made Golden State a 3.5 point favorite at Houston in Game 3 without Curry. This moved to 8.5 points in Game 4 with Curry. Let’s use this to estimate Curry means 5 points per game to the Warriors.

With Steph Curry

With Curry, my numbers give these series win probabilities against the Warriors next two opponents.

  • Los Angeles Clippers: Warriors have a 92.2 percent chance of winning the series.
  • Portland: Warriors have a 95.1 percent chance of winning the series.

Unless the sky falls in, the Warriors should win these series with Curry.

The road gets tougher in the conference finals, but the Warriors should win.

  • San Antonio: Warriors have a 69.5 percent chance of winning the series.
  • Oklahoma City: Warriors have a 80.0 percent chance of winning the series.

Without Steph Curry

Without Curry, the story dramatically changes.

  • Los Angeles Clippers: Warriors have a 69.6 percent chance of winning the series.
  • Portland: Warriors have a 77.7 percent chance of winning the series.

Now, Golden State is vulnerable against either the Clippers or Blazers, even though they should still win.

In the conference finals, the Warriors will no longer be the favorite.

  • San Antonio: Warriors have a 33.9 percent chance of winning the series.
  • Oklahoma City: Warriors have a 46.7 percent chance of winning the series.

NBA championship probabilities

What about the chance of winning an NBA title? With Curry, my numbers give the Warriors a 59.0% chance, which hasn’t changed much since the beginning of the playoffs. Without Curry, their championship probability drops to 16.2%.

Duh, the Warriors aren’t as good without Curry. However, the numbers and market data put in perspective the importance of the NBA’s reigning Most Valuable Player.

How to use analytics to build an NBA champion according to book Chasing Perfection

chasing_perfectionYou love basketball and need an insider look at the NBA.

The stories behind a trade are not enough. It’s 2016, and you know analytics and technology play a huge role in how teams improve their chances at a championship.

In his book Chasing Perfection: A Behind-the-Scenes Look at the High-Stakes Game of Creating an NBA Champion, former Sports Illustrated writer Andy Glockner takes an intimate look at the modern NBA. To the extent that teams will talk, he explores how numbers, health and good old common sense all play a role in the management of a team.

Let’s look at some of the most intriguing stories in the book.

How to build a champion

Golden State has become the marvel of the NBA. The Warriors won the 2015 NBA championship and won a record setting 73 wins the following regular season.

Steph Curry gets most of the headlines because of his ground breaking shooting, and deservedly so. However, Glockner points out that the Warriors “have up to a dozen players with wingspans of nearly seven feet to rotate among the shooting guard, small forward, and power forward positions.”

This length has made the Warriors a sneaky good defensive team. In terms of points allowed per possession, Golden State has ranked first and sixth in 2015 and 2016 respectively.

Chasing Perfection also has another interesting nugget about the Warriors. Assistant GM Kirk Lacob admits that they are “not at the top of the league in terms of either personnel or resources thrown at data analysis.”

And maybe that’s ok. To win a title, it helps to have Steph Curry develop from above average guard to Most Valuable Player. It also doesn’t hurt to have the greatest talent evaluator in NBA history, Jerry West, as an advisor.

How not to build a team

If the Warriors are drinking champagne in the NBA penthouse, the Sixers are crawling through the sewage pipe below the building.

Sam Hinkie brought analytics and new ideas when he became the general manager of the Sixers. As documented by Pablo Torre of ESPN, one of those ideas was to draft players with a long wingspan.

You can teach a player to shoot a basketball, but you can’t teach length. Just like the Warriors, the Sixers collected a stable of long wing players. Then they hoped that these players would make Kawhi Leonard type improvements in shooting.

It didn’t work. The Sixers have been the worst team in the NBA since Hinkie’s arrival. While the defense has been serviceable, the offense has been atrocious.

And it gets worse.

Glockner writes the Sixers took most of their shots from three or near the rim. Since these shots have the highest efficiency, this seems like a good strategy. Not for the Sixers, who have finished a distant last in points scored per possession in each of three seasons of the Hinkie tenure. From Chasing Perfection:

While on a normal team, you might criticize the coach for consistently creating shots his players couldn’t convert, the idea was backwards with the 76ers. (Coach) Brown wanted to create these decent shots, and then have his players (or new ones) learn to make them.

This is utter stupidity. The Sixers set up their players for failure. It’s the opposite of the Spurs strategy, which seeks to maximize the ability of each player.

In April of 2016, Sam Hinkie was fired.

The technology revolution in the NBA

The most interesting part of the book details the technological revolution in player health.

For example, the comany P3 (Peak Performance Project) has an apparatus to measure the force in each leg and make a movie as a player jumps. If you find that one leg produces 20% more force than the other, you predict an injury could be looming for the player.

P3 took their technology to the 2014 NBA pre-draft combine. After testing all the players, they ranked the top 60 players by their likelihood to get hurt. They also predicted the location of the injury.

While Chasing Perfection doesn’t provide any details, P3 said the list “ended up being very predictive.”

P3 also helped Hawks guard Kyle Korver. In a chapter devoted to the sharp shooter, Glockner tells the story of how Korver went from a broke down player who thought about quitting to the healthy, valuable player for the Hawks.

Health analytics played a role in this recovery. P3 showed Korver how one of his knees bowed every time he took a shot. Horrified, Korver got a program to strengthen his body and fix his problem.

This all seems so logical. Players benefit from balance and symmetry in their muscles. With technology, P3 can identify potential problems and get players on a program to fix this muscle imbalance.

The details of P3 also shed light on a cryptic quote from the book Soccernomics
by Simon Kuper and Stefan Szymanski.

AC Milan’s in-house medical outfit found that just by studying a player’s jump, it could predict with 70 percent accuracy whether he would get injured. It then collected millions of data on each of the team’s players on computers, and in the process stumbled upon the secret of eternal youth. (It’s still a secret: no other club has a Milan lab, and the lab won’t divulge its findings.)

There’s no doubt that the Milan lab had similar technology to P3.

Best analytics nugget

Jon Nichols, who now works with the Cleveland Cavaliers, did an interesting study on which college basketball statistics best predict NBA performance. He found that block rate translates best, even more than rebounding and assist rate. The correlation is surprisingly strong.

Best quote

Tom Penn spent 11 years as an NBA executive before moving to ESPN in 2010. He said the following about the adoption of analytics:

Every team over the last fifteen years – doesn’t matter whether they believe in it or not – they do this in order to cover their tail and to demonstrate that they are sophisticated.

Teams do analytics just for show. I wish Penn said things that hilarious and insightful on ESPN.

Most unbelievable story from the book

Buzz Williams, the coach at Virginia Tech, gives his players a weekly talk to make them comfortable with data. The topics range from personal finance to brain science, but they happen each week during the off season.

Holy shit, that’s a huge commitments to the well being of your players. Williams, a long time believer in basketball analytics, may have made a strange move from Marquette to Virginia Tech. However, he deserves recognition as one of the more innovative minds in basketball.


For the hard core hoops junkie, Andy Glockner’s Chasing Perfection gives an inside account of how NBA influencers incorporate data and analytics into their decisions. Of course, it doesn’t tell all, since teams didn’t spill everything to Glockner.

However, the book contains many fascinating anecdotes that reveal the inner workings of basketball. I could have written another thousand words telling the stories like how Ben Alamar’s analytics convinced Oklahoma City to draft Russell Westbrook.

However, it’s best to let Glockner tell those stories in Chasing Perfection.

2016 NBA championship win probabilities

My championship probabilities come from combining rankings that use margin of victory in games and closing point spreads in the markets. In these type of ensemble rankings, the aggregation of different methods tends to cancel out the small mistakes of either method, leading to better predictions.

These numbers come from before the start of the 2016 playoffs.

1. Golden State, 58.0%.
2. San Antonio, 20.1%.
3. Cleveland, 10.7%.
4. Oklahoma City, 5.0%.
5. Toronto, 2.0%.
6. Atlanta, 1.6%.
7. Boston, 0.7%.
8. Los Angeles Clippers, 0.5%.
9. Charlotte, 0.5%.
10. Indiana, 0.4%.
11. Miami, 0.4%.
12. Portland, 0.1%.
13. Detroit, 0.1%.
14. Houston, 0.0%.
15. Dallas, 0.0%.
16. Memphis, 0.0%.

These win probabilities come from the futures markets.

1. Golden State, 53.5%.
2. San Antonio, 16.7%.
3. Cleveland, 16.0%.
4. Oklahoma City, 5.0%.
5. Los Angeles Clippers, 2.0%.
6. Toronto, 1.6%.
7. Miami, 1.1%.
8. Boston, 1.0%.
9. Atlanta, 0.8%.
10. Charlotte, 0.6%.
11. Portland, 0.5%.
12. Indiana, 0.4%.
13. Houston, 0.3%.
14. Detroit, 0.3%.
15. Dallas, 0.2%.
16. Memphis, 0.1%.

There was confusion on Twitter last week about these odds, so let me explain how I came up with them.

Before the playoffs started, I grabbed the futures odds for each team to win the NBA title. These odds translate to a probability for each team.

I find the sum of probabilities for all teams, and then I divide each team’s probability by this normalization factor. This gives an implied probability that factors in the large vig taken by the house (normalization factor was 1.25).

The futures markets like LeBron James and Cavaliers more than the numbers do. This comes at the expense of the Spurs.

2016 NBA playoff series win probabilities for the first round

These numbers come from rankings that use data from games and the markets. To see my numbers for the entire playoffs, check out the interactive visual for NBA win probabilities.

Western Conference

Houston (8) vs Golden State (1).
Golden State has a 96.5 percent chance of winning the series.

Portland (5) vs Los Angeles Clippers (4).
Los Angeles Clippers have a 64.8 percent chance of winning the series.

Dallas (6) vs Oklahoma City (3).
Oklahoma City has a 90.5 percent chance of winning the series.

Memphis (7) vs San Antonio (2).
San Antonio has a 98.4 percent chance of winning the series.

Eastern Conference

Detroit (8) vs Cleveland (1).
Cleveland has a 84.4 percent chance of winning the series.

Boston (5) vs Atlanta (4).
Atlanta has a 60.5 percent chance of winning the series.

Charlotte (6) vs Miami (3).
Miami has a 50.8 percent chance of winning the series.

Indiana (7) vs Toronto (2).
Toronto has a 63.5 percent chance of winning the series.

To see update numbers on these series probabilities, check out the predictions page.