THE POWER RANK

Predictability vs skill in sports analytics: 3 point shooting

By 1 Comment

To listen to the audio version, click on the triangle or grab it on Apple Podcasts. The text version is below.

You’re interested in making sports predictions. The more accurate, the better.

To do this, we typically take a statistic and look at its correlation from season to season. This is especially relevant in preseason football as we look to predict the upcoming season.

If there is a high correlation from season to season, we say the statistic is predictive and include it in our models. If the statistic has a weak correlation, then it’s not predictive. Analytics 101.

However, this exercise can be confusing. For example, consider 3 point shooting percentage in the NBA. There’s a weak correlation from season to season, as a player’s 3 point shooting percentage from last season explains 14.5% of the variance during the current season.

I think there’s a problem with this analysis; more on that later.

In addition, the analysis doesn’t make sense. There is clear skill in shooting a basketball. Who would pick Russell Westbrook over Steph Curry in a three point shooting contest? No one.

In discussing the lack of correlation in 3 point shooting percentage, data scientists like myself usually say something like “randomness plays a big role in 3 point shooting percentage.” This is true, but not the entire picture.

In this article, I’ll discuss predictability versus skill and how they are related but different concepts.

Skill in shooting 3 pointers

Why is there skill in three point shooting?

To get some intuition, let’s take the following approach: assume 3 point shooting is random, and compare the actual data on players with this assumption.

I looked at NBA data from 2014 through 2020 before the lockdown. During this period, the average 3 point shooting percentage was 35.6%. If 3 point shooting is random, each player makes shots at this rate.

Based on this random assumption, a player’s 3 point shooting percentage won’t land exactly on the mean value of 35.6%. Some players will end up above this value, some below. But as a player takes more shots, his shooting percentage approaches the mean value of 35.6%.

When you look at many players based on this random assumption, you get a distribution of 3 point shooting percentages spread around the mean. If each player takes 2000 shots, the width of this distribution is about 1%. This implies that two out of three players will have a percentage between 34.6% and 36.6%.

Let’s compare this random assumption with actual players like Steph Curry. Steph has made 43.2% of his 3 point shots over the past six seasons. If each of Steph’s shots had a 35.6% chance to go in at random, Steph’s shooting percentage would be 9.6 standard deviations from the NBA average (based on 3,681 attempts). This is extremely unlikely.

This confirms what we all know: Steph Curry is a great shooter, probably the best to have ever played the game.

In contrast, Russell Westbrook has made 30.4% of his 2,150 attempts from 3. His percentage is five standard deviations below the NBA average. That’s the kind of ineptitude you expect if you made Dave Gettleman the CTO of your sports analytics startup.

Westbrook is not the worst in the NBA over this period by the standard deviation analysis. More on that later.

These outliers seem to confirm the skill in three point shooting. Let’s put some numbers behind this.

Skill vs luck

To distinguish skill from luck, I’ll use an idea from Michael Maubossian’s book The Success Equation. He defined a model in which outcomes are a combination of skill and luck.

outcome = skill + luck

For 3 point shooting percentage, some of a player’s results comes from skill while the remainder comes from luck. There’s always some randomness. A shot is not always going in, even if Steph Curry is wide open.

Consider the variance in outcomes. Based on this simple model, we get:

Var(outcome) = Var(skill) + Var(luck)

In taking the variance of the previous equation, there is usually a term that considers the correlation between skill and luck. However, by definition, I’m assuming that there’s no correlation between skill and luck. Every player has an equal chance to get lucky.

Let’s go back to our random assumption in which every player makes shots at the same rate. By taking the standard deviation of each player from NBA average, we get a normal distribution with variance of 1. There is no skill in this model.

Let’s compare this assumption with the actual data on NBA players over the past 6 seasons. The wider this distribution in 3 point shooting percentage, the more skill in 3 point shooting.

To measure skill, I consider the fraction of variance in 3 point shooting percentage, Var(outcome), explained by skill, Var(skill). This is similar to the previous idea of predictability. A player’s 3 point shooting percentage from last season explained 14.5% of the variance in a player’s 3 point shooting percentage this season. To get a visual explanation of this concept, click here.

For both predictability and skill, we ask how much of the variance in outcome can be explained by another quantity.

  • For predictability, how does last season’s data explain the variance in this season’s 3 point shooting percentage.
  • For skill, how much bigger is the variance in player 3 point shooting percentage than the variance based on the random assumption.

In the NBA, 78% of the variance in 3 point shooting percentage is explained by skill.

To put this into perspective, let’s look at Maubossian’s results on teams. If winning NFL games were all luck, each game would be a 50-50 coin flip. The distribution of win percentage for teams would have a certain width based on a 16 game season.

The actual distribution of win percentages is wider than the random assumption, and Maubossian calculated the following:

  • NFL: 62% of variance in win percentage is explained by skill
  • NBA: 88% of variance in win percentage is explained by skill

There is more skill in shooting 3 pointers than winning NFL games, as skill explains 78% of the variance in outcomes. However, there is less skill in shooting 3’s than winning NBA games.

For another perspective, let’s look at free throw shooting. Based on the same 6 season NBA data set, skill explains 98% of the variance in free throw shooting percentage.

On the good side, Damian Lillard is 16.8 standard deviations higher than average. On the bad side, Andre Drummond is more than 33 standard deviations worse than NBA average. That is some massive Dave Gettleman ineptitude.

This analysis supports the idea of skill in shooting a basketball. Almost all of the outcome in free throw percentage is skill. It’s the player and the basket. The analysis reveals less skill in 3 point shooting, presumably because of increased randomness due to factors such as defense.

Unlike 3 pointers, free throws are also highly predictable. A player’s free throw shooting percentage from last season explains 70% of the variance in free throw shooting percentage in the current season. We expect that high degree of predictability when a statistic is 98% skill.

3 point shooting percentage is not a strong predictor. From before, a player’s data from last season explains 14.5% of the variance in 3 point shooting percentage this season. This is despite the analysis that 3 point shooting is 78% skill.

3 point shooting is a skill but not predictable. Let’s look at an example.

Unpredictability of 3 point shooting

In performing this analysis, Duncan Robinson of the Miami Heat showed up as one of the best shooters in the NBA. In the 2019-20 season before the lockdown, he made 45% of his 3 point shots.

Robinson played his college basketball at Michigan. He was a senior in the spring of 2018 when Michigan made a run to the NCAA tournament title game against Villanova.

Robinson was a great shooter in college. This was obvious either from looking at his shooting motion or his numbers from his first two seasons at Michigan. But as a senior, he only shot 38% from 3.

However, those 203 eight attempts his senior year didn’t provide a sufficient sample to predict future performance. In his second NBA season, Robinson has shown his skill as a three point shooter.

In contrast, Giannis Antetokounmpo does not have skill in shooting 3 pointers. Over the past 6 seasons, the Greek Freak was the one player worse than Russell Westbrook. Only an MVP caliber player can make 28% of his 3 pointers and still take almost a thousand attempts.

Better 3 point shooting predictions

There’s another problem with looking at year to year correlations in making statements about predictability, especially in pro sports. We have multiple seasons of data on pro players. This is not college.

To see how an increased sample helps predictability, I took the six season NBA sample and asked how five seasons of data could predict the remaining season. To do this, I took a player and picked one of the six seasons at random. To include this player in the analysis, he needed 100 attempts in the target season and 300 in the remaining seasons.

A five year sample was able to explain about 24% of the variance in 3 point shooting percentage in the target season. This doesn’t make me run to put this statistic in a predictive model. However, five seasons gives about a 60% improvement over one season.

For free throw shooting, a five season sample explains 72% of the variance in free throw shooting in the target season. Based on the 70% value from one season, four addition seasons result in about a 3% improvement.

There’s a lot of randomness in three pointers, and a larger sample gives a significant boost in predicting the future. There is less randomness in free throw, and one season is a decent sample to predict the future.

Predictability vs skill in the NFL

Here’s the take home message: predictability and skill are related distinct ideas.

In this analysis, predictability is the correlation of a statistic from an earlier to a later time period. I’m defining skill in terms of the distribution of player statistics over a six year period in the NBA. The variance of this distribution in excess of a random assumption is defined as skill.

Usually, predictability and skill are related. We saw this with free throw shooting. However, skill does not imply predictability. 3 point shooting is a skill, an intuitive results confirmed by the analysis in this article. However, a player’s three point shooting percentage in the past struggles to predict the future.

Next month, we’ll see how these ideas apply to the NFL. In particular, we’ll look at the 32 most important men in sports: NFL quarterbacks.

To make sure you get this analysis, sign up for my free email newsletter. In addition, each week during football season, you’ll get a sample of my best predictions usually saved for paying members of the site.

To sign up, enter your best email and click on “Sign up now!”








Early season NBA rankings for 2018-19

By 1 Comment

I like to look at NBA rankings about this time of the season as we reach a crossover point with football.  NBA, college football and NFL have all played about 10 games.

Here are my NBA team rankings through games on Monday, November 5th, 2018.  The rating gives an expected margin of victory against an average NBA team.

1. Milwaukee, (7-1), 11.57
2. Golden State, (9-1), 9.21
3. Toronto, (9-1), 8.39
4. Denver, (9-1), 7.26
5. Boston, (6-4), 5.45
6. Portland, (7-3), 5.39
7. Charlotte, (4-5), 2.52
8. Indiana, (7-4), 2.30
9. Oklahoma City, (5-4), 1.94
10. Utah, (4-6), 1.25
11. Memphis, (5-4), 1.14
12. New Orleans, (3-6), 1.12
13. Los Angeles Clippers, (5-4), 0.90
14. Los Angeles Lakers, (4-6), 0.63
15. Sacramento, (5-4), -0.09
16. Miami, (4-5), -0.11
17. San Antonio, (6-3), -1.15
18. Minnesota, (4-6), -1.38
19. Houston, (4-4), -1.43
20. Philadelphia, (6-5), -1.52
21. New York, (3-7), -1.64
22. Washington, (2-6), -3.23
23. Orlando, (4-6), -3.48
24. Chicago, (3-7), -4.24
25. Brooklyn, (4-5), -4.58
26. Detroit, (4-5), -5.12
27. Dallas, (2-6), -6.50
28. Phoenix, (2-7), -7.75
29. Cleveland, (1-9), -9.28
30. Atlanta, (3-6), -9.66

From only early season results, we would conclude that top ranked Milwaukee is better than Golden State, the dynasty that has won 3 of the last 4 NBA titles.

We might give up on LeBron, as the Lakers aren’t even the best team in Los Angeles.  The Lakers are ranked 14th, right behind the Clippers.

Of course, these results reflect small sample size.  Remember this when you look only at football numbers from this season. 

Follow along with my NBA rankings, which get updated daily, to see how the season plays out.

Filed Under: NBA

Win probability for the 2018 NBA Finals

By 1 Comment

For the 2018 playoffs, I’ve been using data from game results and markets and then filtering to attempt to account for injuries. More details below.

For the 2018 NBA Finals, my model says that Golden State is better than Cleveland by 6.2 points on a neutral court. This leads to the following odds for the series.

Golden State has a 89.6 percent chance of winning the series.

As of Tuesday afternoon on May 29th, 2018, Bookmaker had Golden State -1100 to win the series (Cleveland is +750). This implies a 88.6% chance to win the series for Golden State once you account for the vig.

I’m really surprised the numbers match up so well. The NBA playoffs have been frustrating to predict from a numbers perspective because of injuries and Golden State’s underachieving. Here’s how my model works.

First, I took the game results from the season and kept only games in which teams had all their key players. For example, Golden State had 38 games in which Steph Curry, Klay Thompson, Kevin Durant and Draymond Green all played.

This reduces the set of games, but it gives a better picture of how a team might perform with its top players. It also assumes Cleveland’s Kevin Love will play. With this reduced set of games, I take margin of victory and adjust for schedule with my ranking algorithm.

Second, I take the closing point spreads in the markets since January 1st and perform the same filtering process with top players. I generate market rankings by adjusting these spreads for schedule.

Then I blended these two rankings to give the following rankings of playoff teams. The rating gives an expected margin of victory against an average NBA team on a neutral court.

1. Golden State, 8.89
2. Houston, 8.11
3. Toronto, 4.59
4. Cleveland, 2.72
5. Philadelphia, 2.32
6. Oklahoma City, 2.17
7. Utah, 2.10
8. Boston, 1.44
9. San Antonio, 1.35
10. Minnesota, 1.26
11. Washington, 1.26
12. Portland, 0.79
13. Indiana, 0.70
14. New Orleans, 0.69
15. Milwaukee, 0.28
16. Denver, 0.26
17. Los Angeles Clippers, -0.60
18. Miami, -1.23
19. Charlotte, -1.39
20. Detroit, -1.46
21. Dallas, -2.71
22. Los Angeles Lakers, -3.68
23. Memphis, -4.27
24. New York, -4.51
25. Orlando, -4.73
26. Brooklyn, -5.14
27. Chicago, -5.44
28. Atlanta, -5.50
29. Phoenix, -6.67
30. Sacramento, -7.35

Filed Under: NBA

2018 NBA Championship Odds at the start of the playoffs

By Leave a Comment

Predicting the 2018 NBA playoffs is a mess. The list of problems starts with these issues:

  • Elite players like Steph Curry and Joel Embiid will start the playoffs injured.
  • Golden State has not performed up to expectation this season, as their defensive efficiency dropped from 2nd to 11th from 2017 to 2018.
  • Kyrie Irving is out for the entire playoffs.

Usually, I take my team rankings that include data from all regular season games and calculate championship probabilities. However, that will not work this season.

Here’s how I approached predicting the 2018 NBA playoffs.

First, I took the game results from the season and kept only games in which teams had all their key players. For example, Golden State had 29 games in which Steph Curry, Klay Thompson, Kevin Durant and Draymond Green all played.

This reduces the set of games, but it gives a better picture of how a team might perform with its top players. These numbers assume Golden State and Philadelphia will have Curry and Embiid respectively. With this reduced set of games, I take margin of victory and adjust for schedule with my ranking algorithm.

Second, I take the closing point spreads in the markets since January 1st and perform the same filtering process with top players. I generate market rankings by adjusting these spreads for schedule.

I decided to focus on the last part of the season to get a more recent picture of team performance. The markets still believe in Golden State, but not as much as in earlier in the season.

Then I blended these two rankings to give the following rankings of playoff teams. The rating gives an expected margin of victory against an average NBA team on a neutral court.

1. Houston, 8.53
2. Golden State, 8.45
3. Toronto, 5.49
4. Oklahoma City, 3.05
5. Cleveland, 2.69
6. Boston, 2.47
7. Utah, 2.46
8. Minnesota, 2.42
9. Philadelphia, 2.05
10. Washington, 1.93
11. San Antonio, 1.79
12. Portland, 1.56
14. Indiana, 0.70
15. New Orleans, 0.57
16. Milwaukee, 0.41
18. Miami, -0.48

This leads to the following championship probabilities for 2018 NBA playoffs.

1. Houston, 39.4%
2. Golden State, 36.5%
3. Toronto, 14.9%
4. Boston, 2.2%
5. Cleveland, 1.9%
6. Philadelphia, 1.6%
7. Oklahoma City, 0.9%
8. Washington, 0.5%
9. Utah, 0.5%
10. Portland, 0.4%
11. Minnesota, 0.3%
12. San Antonio, 0.2%
13. Indiana, 0.2%
14. Milwaukee, 0.2%
15. New Orleans, 0.1%
16. Miami, 0.1%

You can compare the calculations with these implied odds from the markets, taken from Bookmaker on Friday morning, April 13th, 2018.

1. Golden State, 36.6%
2. Houston, 28.9%
3. Cleveland, 10.9%
4. Toronto, 9.1%
5. Philadelphia, 4.8%
6. Oklahoma City, 2.1%
7. Utah, 1.7%
8. Portland, 1.4%
9. San Antonio, 1.2%
10. Boston, 1.0%
11. Indiana, 0.5%
12. Washington, 0.5%
13. Milwaukee, 0.4%
14. Miami, 0.4%
15. Minnesota, 0.3%
16. New Orleans, 0.3%

Houston and Golden State should have the best odds to win the title, and both the calculations and markets agree on this.

My calculations are low on Cleveland. The conventional wisdom is that regular season results don’t matter in predicting how LeBron James will perform in the playoffs. This may or may not be true.

However, LeBron does play almost 4 more minutes per game in the playoffs than in the regular season. This should make Cleveland better in the playoffs. My numbers don’t account for different minute distribution and shows how better models could be built with play by play data.

My calculations are also too high on Boston. I did not exclude games in which Kyrie Irving played in my analysis, so some adjustment should be made for this injury.

Get the game by game predictions each day on the main predictions page.

Filed Under: NBA

2017 NBA Finals Series win probability

By 1 Comment

I strongly like Golden State to win the NBA Finals over Cleveland. Let me back that up with some numbers.

My best NBA projections use rankings that incorporate both scores of games and closing point spreads from the markets. These ensemble rankings rate Golden State 5.3 points better than Cleveland on a neutral court.

I then calculate a series win probability by considering this rating differential and a home court advantage of 4.5 points. This implies a 86.3% win probability for Golden State.

It’s worth breaking down some of the components of these ensemble rankings.

By team rankings that consider all games, Golden State rates 7.1 points better than Cleveland on a neutral court. However, this is a misleading number as teams rest their star players and deal with injuries.

I adjust for these player absences by only considering games in which a team has its top 2 or 3 players. These filtered team rankings make Golden State 4.0 points better than Cleveland, which corresponds to a 80.5% win probability.

I like this number since it closely corresponds with another set of my calculations.

In my market rankings, I take closing lines and adjusting for opposition with my ranking algorithm. The playoffs markets have been distorted by injuries, so I use data from the trade deadline in February until the end of the regular season.

These market rankings make Golden State 4.6 points better than Cleveland on a neutral court. This corresponds to a 83.5% series win probability for the Warriors.

Each of these components gives a win probability for Golden State higher than the 73% implied by the markets (Golden State -270 on Wednesday at Bookmaker). The markets feel like Cleveland and LeBron James can outplay their year long numbers more than Golden State.

I don’t buy it, and it comes down to defense.

In points allowed per possession, Cleveland ranked 21st this season, a poor showing for a championship caliber team. This group is capable of better, as they ranked 10th last season.

However, their defense isn’t nearly as good as that of Golden State, which ranked 2nd this season after 5th last season. It will make the difference in this series, which could be much shorter than anyone expects.

Filed Under: NBA