This work is adapted from The Pick Report, my full study on how to predict interceptions. To check out the full product, click here.
Is Carson Wentz skilled at not throwing interceptions?
The raw data on the Eagles quarterback certainly supports this claim. While the NFL average interception rate has been 2.4% the past three seasons, Wentz has been better:
- 2017 – 1.6%
- 2018 – 1.7%
- 2019 – 1.1%
Over these years, Wentz had his best season in 2017 despite not having his best interception rate. The former #2 pick overall led Philadelphia to an 11-2 start before an injury derailed his season. Nick Foles finished the job as the Eagles won the Super Bowl.
Wentz had his best season of interception prevention in 2019. However, the offense struggled as he dealt with injuries to his receivers.
Will Wentz’s ability to prevent interceptions continue? Unfortunately, we can’t make any statements based on his past interception rate. The data shows very little predictability for interceptions, a concept I’ll make precise in the next section.
To predict interceptions, we need to first check the skill in throwing interceptions and then dig deeper into the data on NFL quarterbacks. Let me explain.
The predictability of interceptions
Why are interceptions difficult to predict?
To look at the most basic idea of predictability, consider a quarterback’s interception rate, or interceptions divided by pass attempts. We ask how sticky this quarterback statistic is from season to season.
Stickiness is determined by calculating the correlation of interception rate from one season to the next. This analysis considers the 2000 through 2019 seasons, and a quarterback gets included if he has thrown 200 pass attempts in consecutive seasons.
Interception rate the previous season explains only 7% of the variance in interception rate during the current season. To understand this result, think of a plot of data points that correspond to consecutive seasons. The interception rate from the previous season is on the x-axis, or the horizontal axis. The interception rate from the current season, or the variable we would like to predict, is on the y-axis, or the vertical axis.
If interception rate were a good predictor, the data points would hug a line. There would be scatter around the line, since no predictor is perfect. But the data would suggest a clear relationship between the two quantities.
There is no clear relationship between interception rate in consecutive seasons. Interception rate from the previous season explains 7% of the variance in the current season, and this 7% is known as the R-squared value. The plot of the data looks like a Jackson Pollock painting.
But are interceptions really that random and unpredictable? To think about this in a different way, let’s ask about the relative contribution of skill and luck.
The skill in interceptions
To understand how this works, let’s first assume there is zero skill in throwing interceptions. This means that every quarterback has a 2.4% chance of throwing an interception on a pass attempt. If this assumption matches up with data on players, then interceptions would be all luck.
However, this seems improbable. Consider Jameis Winston. Tampa Bay fans had high hopes when he was drafted as the top pick in the 2015 draft. They had even higher hopes in 2019 when Bruce Arians arrived as his head coach.
However, Jameis was a turnover machine in 2019, as he threw interceptions on 4.8% of his pass attempts. This percentage was a little bit of an anomaly. In his career before Arians, he had a 3% interception rate. That’s not good, but it’s not atrocious like his 2019 rate.
Over his career, Jameis has thrown 2,551 pass attempts. If interceptions were all luck, we would expect his interception rate to be close to the NFL average of 2.4%. With more pass attempts, the more likely his interception rate is close to this average. Remember, the assumption is that interceptions are all luck. This makes the math easy, as the distribution of Jameis’ interception rate is something called a binomial distribution.
With enough pass attempts, this distribution looks like a normal distribution, or the bell curve that you often see. For Jameis, this bell curve implies a 2 in 3 chance his interception rate is between 2.1% and 2.7%. This means the width, or standard deviation, is 0.3%. If Jameis threw another two thousand pass attempts, the width would get smaller.
Jameis’ actual interception rate is higher than expected from the randomness assumption. During his career, he’s averaged a 3.5% interception rate. Based on the width of 0.3%, Jameis is three and a half standard deviations worse than NFL average. That’s as bad as the acting of Hayden Christiansen as Anakin Skywalker in Star Wars.
How about the new quarterback in Tampa Bay? Tom Brady has thrown picks on 1.4% of his pass attempts over the last six seasons. Based on the randomness assumption and 5,500 pass attempts, he is four standard deviations better than the NFL average.
These extremes in interception rate suggest a contribution of skill to interceptions. To quantify this, I’m going to use an idea from Michael Maubossian in his book The Success Equation. He said that:
Outcome = Skill + Luck
Here, the outcome is interception rate. The skill part is related to the accuracy of the quarterback. A player who can regularly throw a football through a tire 50 yards away will throw fewer interceptions. Aaron Rogers has a 1.1% interception rate since the 2014 season.
A rate that low might not be the best idea. Minimizing interception rate should not be the goal in football. The goal should be to win football games. If a team is down late in the game and needs a touchdown, the optimal strategy is throwing to Davante Adams in double coverage, even though there’s a higher chance of a pick. This is better than dumping the ball off to a running back.
In addition to skill, there is luck involved in throwing interceptions. Even the most accurate quarterback will throw an errant pass at times. In addition, the speed of a modern NFL defense forces some tipped passes and pass break ups. The offense has no control over the trajectory of the ball once these plays happen. Despite their best efforts, Tom Brady and Aaron Rogers will never have a zero interception rate.
With the Maubossian model, let’s consider the variance of the outcome, or the variance in interception rate. This is given by the following:
Var(Outcome) = Var(Skill) + Var(Luck)
Usually when we take the variance of a linear equation, there’s a term that considers the correlation between skill and luck. However, by definition, there is no correlation between skill and luck. Luck, good or bad, does not discriminate between Jameis Winston and Tom Brady.
Before, we looked at the deviation in interception rate for Jameis Winston and Tom Brady. Let’s do the same for all quarterbacks the past six seasons. Based on the randomness assumption, we calculate a standard deviation from NFL average. This depends on the number of pass attempts, and the analysis only includes quarterbacks with 500 pass attempts during this time period.
If interceptions were all luck, this player data would produce a bell curve with a width of one. The preliminary analysis suggested a distribution with a larger width. The wider this distribution, the more skill in interceptions.
The key quantity is the fraction of variance in outcome that comes from variance in skill. This is similar to what we looked at with predictability.
With predictability, we were interested in the interception rate for a single season. We reported how much of the variance in this interception rate was explained by interception rate the previous season.
With skill, we consider a player’s interception rate over six seasons. We will report how much of the variance in this interception rate is explained by skill. There would be some variance in interception rate due to randomness. The simple Maubossian model attributes the excess variance to skill.
Based on the past six seasons, interception rate is 66% skill.
Interception rate has low predictability. The interception rate last season explains 7% of the variance in interception rate this season. However, it’s 66% skill.
We see the same trend with three point shooting in the NBA. For predictability, last season’s three point shooting percentage for a player explains 14.5% of the variance in this year’s three point shooting percentage. However, three point shooting is 78% skill.
Interceptions and three point shooting have low predictability but high skill.
This skill in interceptions suggests we should be able to do better at predicting interception rate. To do this, we need to consult a different variable.
A hidden variable
To better predict interceptions, let’s consult an idea first introduced by Bill Connelly during his time at SB Nation and expanded upon by Aaron Schatz of Football Outsiders.
To understand this idea, consider passes defended. An incomplete pass is considered to be defended if one of three events happens:
- The pass gets tipped at the line of scrimmage.
- The ball gets deflected by a defender in coverage.
- A defender hits the receiver as the ball arrives and jars the ball loose.
These three events get counted as passes defended. In looking at a quarterback, we can track how often he puts a ball in a dangerous situation through these passes defended.
Consider the rate at which a quarterback allows the defense to break up a pass. His passes defended rate is passes defended divided by pass attempts.
First, let’s ask about the stickiness of this quantity from year to year. For quarterbacks, last year’s passes defended rate explains 20% of the variance in passes defended rate during the current season. That’s almost three times as much variance explained as interception rate.
What about skill versus luck in passes defended? Over the past six seasons, 75% of passes defended rate is explained by skill. This is more than the 66% skill with interception rate but less than the 86% skill with completion percentage.
The idea with passes defended is that a certain fraction of these balls will end as interceptions. In an ideal world, this fraction should not be sticky from season to season. We expect strong regression to the mean. This is Bill Connelly’s original analysis in college football.
To check this, consider interceptions as a fraction of bad balls. Bad balls are all passes in which a quarterback puts the ball in a dangerous situation. This includes interceptions and passes defended.
Now consider the fraction of bad balls that end up as interceptions. To calculate this, take interceptions and divide by interceptions plus passes defended. In the NFL from 2014 to 2019, 21.3% of bad balls ended up as interceptions.
This percentage of interceptions to bad balls is not sticky at all. There is zero correlation from season to season. This means we can expect strong regression to the mean from year to year.
We should also check the skill in this rate of interceptions to bad balls. The analysis reveals this rate is 29% skill. The rate of interceptions to bad balls is not all random. Compared to the NFL average of 21.3%, Jameis Winston is still at one extreme with a 27.6% pick to bad ball rate. Aaron Rodgers is at the other extreme with a 14.3% rate, but Tom Brady appears closer to the mean at 19.0%.
In the end, the skill in interceptions per bad balls at 29% is way less than the skill in fumble rate on sacks, which is 49% skill. In addition, with the low level of predictability, the picks to bad ball rate should regress to the mean for quarterbacks.
Carson Wentz in 2020
Will Carson Wentz continue to prevent interceptions? The data suggests no.
In 2019, 10.5% of his bad balls ended up as interceptions. This quantity shows strong regression to the mean, which implies that Wentz’s interception to bad ball rate will be closer to the NFL average of 21.3% in 2020.
It is not likely Wentz will have an interception rate as low as 1.1% again like he did in 2019. He will be closer to the NFL average of 2.4%.
This analysis was adapted from The Pick Report, my study on how to predict interceptions. To get the full report, click on “Add to Cart.”
First of all, what nerve, denigrating Mr. Christiansen’s masterful protrayal of a young Vader, which made The Bard himself weep from beyond the grave! No thespian before or since has so movingly conveyed the coarseness of sand, the atrocities of sand people, or the torment of the soul from unrequited love. I pray thee revisit him.
Secondly, have you broken these passes up by distance? I’m sure the mad bombers have a higher percentage of passes picked off. Cian Fahey has done some great work on this with his “interceptable pass” metric. A lot of interception rate variance comes from the sheer randomness of D-backs managing/failing to hold on to the balls that hit their hands.
Regardless, it’s a fun topic, ripe with research potential! I’m sure Hayden could deliver an Oscar-worthy soliloquy on the vagaries of year-over-year interception rates.
I loved this post. By sharing the thought process and some measure of detail in your analysis, you provide visibility into key assumptions and limitations of your research. That does not invalidate your findings, it helps me to better understand what your findings actually appear to mean (and not necessarily mean as well). Too often, analytics “experts” (like Cynthia Frelund who is probably a great statistician and researcher) leave these details undisclosed, making it impossible for a reasonable fan to draw conclusions about the scope (and potential limitations) of her inferences. Great work!