On this episode of The Football Analytics Show, I’m joined by Dr. Eric Eager, the Executive Director of Research and Development at Pro Football Focus. Among other topics, we discuss:

• How PFF uses grades for the betting markets (3:06)
• How PFF grades deal with cornerbacks (7:08)
• The most important research PFF did this off season (10:48)
• The insights from the PFF QB Annual, and two QBs to fade based on this data (15:56)
• A bold prediction for the NFL in 2020 (36:02)

We also broke down a number of NFL teams:

• San Francisco and fragility (23:09)
• Kansas City and why not to go against them (27:52)
• Green Bay and Minnesota based on week 1 (30:08)
• Los Angeles Rams and the defense (31:58)
• Philadelphia (33:20)
• Tennessee (36:02)
• Pittsburgh (38:10)

Dr. Eager is a superstar in combining data and watching football, and you’ll get a ton out of his insights during this episode.

To listen, click on the right pointing triangle.

To listen on Apple Podcasts, click here.

The Football Analytics Show is also available on Spotify, Stitcher and Google Podcasts.

This work is adapted from The Pick Report, my full study on how to predict interceptions. To check out the full product, click here.

Is Carson Wentz skilled at not throwing interceptions?

The raw data on the Eagles quarterback certainly supports this claim. While the NFL average interception rate has been 2.4% the past three seasons, Wentz has been better:

• 2017 – 1.6%
• 2018 – 1.7%
• 2019 – 1.1%

Over these years, Wentz had his best season in 2017 despite not having his best interception rate. The former #2 pick overall led Philadelphia to an 11-2 start before an injury derailed his season. Nick Foles finished the job as the Eagles won the Super Bowl. 

Wentz had his best season of interception prevention in 2019. However, the offense struggled as he dealt with injuries to his receivers.

Will Wentz’s ability to prevent interceptions continue? Unfortunately, we can’t make any statements based on his past interception rate. The data shows very little predictability for interceptions, a concept I’ll make precise in the next section.

To predict interceptions, we need to first check the skill in throwing interceptions and then dig deeper into the data on NFL quarterbacks. Let me explain.

The predictability of interceptions

Why are interceptions difficult to predict?

To look at the most basic idea of predictability, consider a quarterback’s interception rate, or interceptions divided by pass attempts. We ask how sticky this quarterback statistic is from season to season.

Stickiness is determined by calculating the correlation of interception rate from one season to the next. This analysis considers the 2000 through 2019 seasons, and a quarterback gets included if he has thrown 200 pass attempts in consecutive seasons.

Interception rate the previous season explains only 7% of the variance in interception rate during the current season. To understand this result, think of a plot of data points that correspond to consecutive seasons. The interception rate from the previous season is on the x-axis, or the horizontal axis. The interception rate from the current season, or the variable we would like to predict, is on the y-axis, or the vertical axis.

If interception rate were a good predictor, the data points would hug a line. There would be scatter around the line, since no predictor is perfect. But the data would suggest a clear relationship between the two quantities.

There is no clear relationship between interception rate in consecutive seasons. Interception rate from the previous season explains 7% of the variance in the current season, and this 7% is known as the R-squared value. The plot of the data looks like a Jackson Pollock painting.

But are interceptions really that random and unpredictable? To think about this in a different way, let’s ask about the relative contribution of skill and luck.

The skill in interceptions

To understand how this works, let’s first assume there is zero skill in throwing interceptions. This means that every quarterback has a 2.4% chance of throwing an interception on a pass attempt. If this assumption matches up with data on players, then interceptions would be all luck.

However, this seems improbable. Consider Jameis Winston. Tampa Bay fans had high hopes when he was drafted as the top pick in the 2015 draft. They had even higher hopes in 2019 when Bruce Arians arrived as his head coach.

However, Jameis was a turnover machine in 2019, as he threw interceptions on 4.8% of his pass attempts. This percentage was a little bit of an anomaly. In his career before Arians, he had a 3% interception rate. That’s not good, but it’s not atrocious like his 2019 rate.

Over his career, Jameis has thrown 2,551 pass attempts. If interceptions were all luck, we would expect his interception rate to be close to the NFL average of 2.4%. With more pass attempts, the more likely his interception rate is close to this average. Remember, the assumption is that interceptions are all luck. This makes the math easy, as the distribution of Jameis’ interception rate is something called a binomial distribution.

With enough pass attempts, this distribution looks like a normal distribution, or the bell curve that you often see. For Jameis, this bell curve implies a 2 in 3 chance his interception rate is between 2.1% and 2.7%. This means the width, or standard deviation, is 0.3%. If Jameis threw another two thousand pass attempts, the width would get smaller. 

Jameis’ actual interception rate is higher than expected from the randomness assumption. During his career, he’s averaged a 3.5% interception rate. Based on the width of 0.3%, Jameis is three and a half standard deviations worse than NFL average. That’s as bad as the acting of Hayden Christiansen as Anakin Skywalker in Star Wars. 

How about the new quarterback in Tampa Bay? Tom Brady has thrown picks on 1.4% of his pass attempts over the last six seasons. Based on the randomness assumption and 5,500 pass attempts, he is four standard deviations better than the NFL average. 

These extremes in interception rate suggest a contribution of skill to interceptions. To quantify this, I’m going to use an idea from Michael Maubossian in his book The Success Equation. He said that:

Outcome = Skill + Luck

Here, the outcome is interception rate. The skill part is related to the accuracy of the quarterback. A player who can regularly throw a football through a tire 50 yards away will throw fewer interceptions. Aaron Rogers has a 1.1% interception rate since the 2014 season. 

A rate that low might not be the best idea. Minimizing interception rate should not be the goal in football. The goal should be to win football games. If a team is down late in the game and needs a touchdown, the optimal strategy is throwing to Davante Adams in double coverage, even though there’s a higher chance of a pick. This is better than dumping the ball off to a running back. 

In addition to skill, there is luck involved in throwing interceptions. Even the most accurate quarterback will throw an errant pass at times. In addition, the speed of a modern NFL defense forces some tipped passes and pass break ups. The offense has no control over the trajectory of the ball once these plays happen. Despite their best efforts, Tom Brady and Aaron Rogers will never have a zero interception rate. 

With the Maubossian model, let’s consider the variance of the outcome, or the variance in interception rate. This is given by the following:

Var(Outcome) = Var(Skill) + Var(Luck)

Usually when we take the variance of a linear equation, there’s a term that considers the correlation between skill and luck. However, by definition, there is no correlation between skill and luck. Luck, good or bad, does not discriminate between Jameis Winston and Tom Brady. 

Before, we looked at the deviation in interception rate for Jameis Winston and Tom Brady. Let’s do the same for all quarterbacks the past six seasons. Based on the randomness assumption, we calculate a standard deviation from NFL average. This depends on the number of pass attempts, and the analysis only includes quarterbacks with 500 pass attempts during this time period.

If interceptions were all luck, this player data would produce a bell curve with a width of one. The preliminary analysis suggested a distribution with a larger width. The wider this distribution, the more skill in interceptions. 

The key quantity is the fraction of variance in outcome that comes from variance in skill. This is similar to what we looked at with predictability. 

With predictability, we were interested in the interception rate for a single season. We reported how much of the variance in this interception rate was explained by interception rate the previous season.

With skill, we consider a player’s interception rate over six seasons. We will report how much of the variance in this interception rate is explained by skill. There would be some variance in interception rate due to randomness. The simple Maubossian model attributes the excess variance to skill.

Based on the past six seasons, interception rate is 66% skill.

Interception rate has low predictability. The interception rate last season explains 7% of the variance in interception rate this season. However, it’s 66% skill.

We see the same trend with three point shooting in the NBA. For predictability,  last season’s three point shooting percentage for a player explains 14.5% of the variance in this year’s three point shooting percentage. However, three point shooting is 78% skill. 

Interceptions and three point shooting have low predictability but high skill.

This skill in interceptions suggests we should be able to do better at predicting interception rate. To do this, we need to consult a different variable.

A hidden variable

To better predict interceptions, let’s consult an idea first introduced by Bill Connelly during his time at SB Nation and expanded upon by Aaron Schatz of Football Outsiders.

To understand this idea, consider passes defended. An incomplete pass is considered to be defended if one of three events happens:

1. The pass gets tipped at the line of scrimmage. 
2. The ball gets deflected by a defender in coverage.
3. A defender hits the receiver as the ball arrives and jars the ball loose. 

These three events get counted as passes defended. In looking at a quarterback, we can track how often he puts a ball in a dangerous situation through these passes defended. 

Consider the rate at which a quarterback allows the defense to break up a pass. His passes defended rate is passes defended divided by pass attempts. 

First, let’s ask about the stickiness of this quantity from year to year. For quarterbacks, last year’s passes defended rate explains 20% of the variance in passes defended rate during the current season. That’s almost three times as much variance explained as interception rate. 

What about skill versus luck in passes defended? Over the past six seasons, 75% of passes defended rate is explained by skill. This is more than the 66% skill with interception rate but less than the 86% skill with completion percentage. 

The idea with passes defended is that a certain fraction of these balls will end as interceptions. In an ideal world, this fraction should not be sticky from season to season. We expect strong regression to the mean. This is Bill Connelly’s original analysis in college football.

To check this, consider interceptions as a fraction of bad balls. Bad balls are all passes in which a quarterback puts the ball in a dangerous situation. This includes interceptions and passes defended. 

Now consider the fraction of bad balls that end up as interceptions. To calculate this, take interceptions and divide by interceptions plus passes defended. In the NFL from 2014 to 2019, 21.3% of bad balls ended up as interceptions. 

This percentage of interceptions to bad balls is not sticky at all. There is zero correlation from season to season. This means we can expect strong regression to the mean from year to year. 

We should also check the skill in this rate of interceptions to bad balls. The analysis reveals this rate is 29% skill. The rate of interceptions to bad balls is not all random. Compared to the NFL average of 21.3%, Jameis Winston is still at one extreme with a 27.6% pick to bad ball rate. Aaron Rodgers is at the other extreme with a 14.3% rate, but Tom Brady appears closer to the mean at 19.0%. 

In the end, the skill in interceptions per bad balls at 29% is way less than the skill in fumble rate on sacks, which is 49% skill. In addition, with the low level of predictability, the picks to bad ball rate should regress to the mean for quarterbacks.

Carson Wentz in 2020

Will Carson Wentz continue to prevent interceptions? The data suggests no.

In 2019, 10.5% of his bad balls ended up as interceptions. This quantity shows strong regression to the mean, which implies that Wentz’s interception to bad ball rate will be closer to the NFL average of 21.3% in 2020.

It is not likely Wentz will have an interception rate as low as 1.1% again like he did in 2019. He will be closer to the NFL average of 2.4%.

This analysis was adapted from The Pick Report, my study on how to predict interceptions. To get the full report, click on “Add to Cart.”

Add to Cart.