The surprising truth about passing and rushing in the NFL

By 3 Comments

Before the 1991 Super Bowl, Bill Belichick told his Giants defense to let Buffalo running back Thurman Thomas rush for 100 yards.

As David Halberstam writes in Education of a Coach, it was a tough sell. The New York Giants played a physical defense that prided itself on not allowing 100 yard rushers.

No matter, the short, stout coach looked straight into the eyes of Lawrence Taylor and Pepper Johnson and said, “You guys have to believe me. If Thomas runs for a hundred yards, we win this game.”

Just in case his players didn’t listen, Belichick took it upon himself to ensure Thomas got his yards. He took out a defensive lineman and linebacker and replaced those large bodies with two defensive backs. In football lingo, the Giants played a 2-3-6 defense designed to struggle against the run.

Did Bill Belichick go insane? I certainly thought so when I first read this story years ago.

However, analytics is on Belichick’s side. Let me explain.

The truth about rushing in the NFL

To examine Belichick’s strategy, let’s use analytics to evaluate the importance of rushing compared to passing in the NFL.

To make this comparison, we must move past misleading statistics like rush yards per game. Teams with the lead run the ball to take time off the clock. Any team can rush for 100 yards if they run it 50 times.

To measure true skill, let’s consider yards per attempt, a powerful efficiency metric. A team can’t fake their way to 5 yards per carry by running the ball more.

In the NFL, I’ll define team efficiency for passing and rushing as yards gained per attempt on offense minus yards allowed per attempt on defense. Better offenses gain more yards per attempt. However, better defenses allow fewer yards per attempt, so subtracting these smaller values leads to higher team efficiency.

Team efficiency can come from the offense, defense, or both. For example, consider Carolina in 2015, the season they played in the Super Bowl. The offense gained 6.2 yards per attempt, 13th best in the NFL. This rank might seem low given that quarterback Cam Newton won Most Valuable Player award that season. The defense allowed 5.0 yards per attempt, 2nd best. This gives a team pass efficiency of 1.2 yards per attempt, 5th in the NFL.

In these numbers, sacks count as pass attempts. These negative yards count against the yards per attempt for an offense. In rush efficiency, the value comes directly from yards per carry.

To examine the relative importance of passing versus rushing, let’s look at how playoff and Super Bowl winning teams performed in team efficiency. Strong teams in passing and rushing will have positive efficiency values, or better than the NFL average of zero.

NFL playoff teams excel in passing, either by throwing the ball on offense, preventing the pass on defense, or both. From 1998 through 2017, only 39 of 252 playoff teams allowed more yards per pass attempt than they gained. This implies that 84.5% of playoff teams had a positive pass efficiency.

Super Bowl champions excelled in pass efficiency as well, as 15 of the 21 champions had pass efficiencies of a yard per attempt or more.

However, excellence in the air does not guarantee success in the small sample size of the playoffs. Three Super Bowl champions had negative pass efficiency during the regular season. Let’s look more closely at two of these Super Bowls.

  • After the 2007 season, the New York Giants beat New England, a team with Tom Brady and Randy Moss looking for an undefeated season.
  • After the 2001 season, New England beat St. Louis, the Greatest Show on Turf led by Kurt Warner and Marshall Faulk.

These are considered two of the greatest Super Bowl upsets ever.

The importance of the passing game in the NFL should not be a surprise. Quarterbacks dominate the headlines and earn the highest salaries. With the importance of throwing the ball on offense, it makes sense that teams with a strong pass defense also excel.

However, the insignificance of rushing in the NFL might surprise you.

From 1997 through 2017, only 57.5% of playoff teams (145 of 252) had a positive team rush efficiency. The visual of rush efficiency for playoff teams shows a random scatter of points with both positive and negative values. A strong run game or stout rush defense has little effect in helping an NFL team win enough games to make the playoffs.

Moreover, running the football doesn’t suddenly become important in the playoffs. If it did, the teams with positive values of team rush efficiency would make the Super Bowl. Instead, only 26 of 42 Super Bowl teams from 1997 through 2017 had a positive rush efficiency value, a rate of 61.9%.

For example, Indianapolis won the Super Bowl after the 2006 season despite having the NFL’s worst team rush efficiency during the regular season. Green Bay ranked 31st out of 32 teams in rush efficiency during the 2010 regular season, but the Packers won the Super Bowl anyway.

These teams had elite quarterbacks, as Peyton Manning of Indianapolis and Aaron Rodgers for Green Bay propelled these teams to regular and post season success. We’ll get back to these two later when we explore whether a team needs a run game to throw the ball effectively.

These raw yards per carry numbers can get skewed a bit. Teams that excel in pass efficiency will tend to have the lead late in games. In these situations, they will run the ball more to take time off the clock. The defense expects the run, which should hinder their rush efficiency.

From 2012 through 2016, teams up 7 or more points in the fourth quarter rushed for 3.29 yards per carry, well below the NFL average of 4.18. While this certainly hurts the rush efficiency of good teams, these carries only make up 12.0% of all carries during this period.

Teams down 7 or more points in the fourth quarter rush for 4.91 yards per carry, so these bad teams get a bump in rush efficiency. However, these carries only make up 5.1% of all NFL carries, as these teams opt to throw the ball as the best means to get back into the game.

Excluding these carries would not change the main claim about the insignificance of rushing in the NFL.

Sign up for football predictions

Hope you’re enjoying the article so far. You have a ways to go, as we haven’t even discussed whether you need the run to set up play action.

If you sign up for my free email newsletter, I’ll send a paper copy of this manifesto to read at your leisure. You’ll also get a weekly sample of football predictions usually reserved for paying members of the site.

To get this free service, enter your best email and click on “Sign up now!”









A guessing game of a team’s wins

In the NFL, running the ball does not affect winning as much as you think. To illustrate this point, consider this guessing game. Suppose you want to guess how many games a team will win during the regular season. Without any other data, it makes sense to guess 8, the average number of wins in a 16 game season.

From 1997 through 2016, this estimate would be wrong by 3.08 wins. In technical jargon, 3.08 is the standard deviation of actual wins from the guess of 8. In normal people language, this means 2 of 3 teams will be within 3.08 wins of the guess. About two thirds of NFL teams won between 5 and 11 games between 1997 and 2016.

How much better does your guess get knowing the rush efficiency for each team? To determine the correction, you plot wins as a function of team rush efficiency and draw a best fit line through the data. The bottom panel of the visual shows this relationship for all NFL teams from 1997 through 2016.

The regression line informs a new guess about the number of games a team will win. For example, suppose a team has a rush efficiency of 0.6 yards per carry. Instead of guessing 8 wins for this team, the line shows 8.7 wins for this team.

How much better are these new guesses? Not much. The error only drops from 3.08 wins to 3.01 wins. In technical jargon, rush efficiency explains only 5.0% of the variance in wins. (For a simple, visual explanation of the previous statement, click here.) You might as well guess at random. 

The results of the guessing game get better with team pass efficiency, as the visual shows that better passing teams tend to win more games. The error in estimating wins drops from 3.08 to 1.95. Pass efficiency explains 60.2% of the variance in wins in the NFL.

In college football, rush efficiency correlates more strongly with wins than in the NFL. Teams like Alabama, Stanford and Wisconsin have won with a power running game and a physical front seven on defense. This level of insignificance in the rushing game is unique to the NFL.

To put these numbers in context, let’s look at two NFL franchises and how their fate depended on the efficiency with which they threw or ran the ball.

Indianapolis had a remarkable run from 2003 through 2010. Under the leadership of General Manager Bill Polian and quarterback Peyton Manning, the Colts won at least 12 games each year before slacking off with 10 wins in 2010.

The Colts achieved success through the air, ranking in the top 8 in team pass efficiency each year. Peyton Manning and his offense played the bigger role, but the pass defense helped out some years. The Colts ranked in the top 10 in yards allowed per pass attempt from 2007 through 2009.

However, Indianapolis was really bad in the run game. Only once in this era (2007) did they gain more yards per carry than they allowed.

In 2006, they had the worst team rush efficiency in the NFL, as their rush defense allowed a league worst 5.3 yards per carry. The rush defense performed better in the playoffs, as they allowed a near NFL average 4.1 yards per carry. However, Manning and the pass offense led Indianapolis to their Super Bowl win that year over Chicago.

In contrast to Indianapolis, Minnesota dominated the ground game during this same period. They featured running back Adrian Peterson on offense and had tackles Pat and Kevin Williams clogging up the middle on defense. From 2006 through 2013, Minnesota finished 1st in rush efficiency four times and 3rd another two times.

However, this strength led to ups and downs in wins. Minnesota went 3-13 in 2011 despite leading the NFL in rush efficiency. The next season, they led the NFL again behind a monster season from Peterson, who made a remarkable return from knee surgery. The Vikings had a 10-6 record that season.

The Viking’s best season over this stretch came in 2009. They finished 12th in rush efficiency that season, by far their worst rank. The difference? A quarterback named Brett Farve came out of retirement to play for Minnesota. The Vikings finished 7th in yards gained per pass attempt, went 12-4 and came within a late turnover against New Orleans of playing in the Super Bowl.

While these two franchises represent the extremes, the trend is clear. In the modern NFL, teams that can throw the ball and defend the pass win football games and championships. It matters little whether they can run the ball or stop the run.

Will this trend continue into the future? Will the NFL continue to prize the quarterback, cornerbacks and pass rushers? Or will running backs make a come back soon?

To guess at the future, we first need to look at the past and the evolution of the passing game.

Bill Walsh and the West Coast Offense

In the late 1960’s, Cincinnati’s offensive coordinator Bill Walsh had a problem: his players sucked.

As Michael Lewis describes in his book Blindside, the quarterback couldn’t throw the ball more than 20 yards down the field. The receivers defined the term “replacement level.” They may have struggled in a flag football game.

With this lack of talent, Walsh had to innovate.

Instead of a deep passing game that required a strong armed quarterback and fast receivers, he developed a passing game based on short drop backs by the quarterback and well-timed passes to the receivers. The ball traveled a shorter distance in the air, making an interception less likely. If the quarterback could hit the receiver in stride, the offense might gain as many yards after the catch as in the air before it.

Walsh got many of these ideas from San Diego offensive coordinator Sid Gilman, but he put his own spin on the West Coast offense. Soon, he coaxed quarterback Virgil Carter into leading the NFL in completion percentage in 1971.

Then Walsh made a pleasant discovery. This West Coast offense didn’t just work when your players sucked. It kept moving the ball efficiently with better players. Cincinnati made the AFC championship game in 1975 with Kenny Anderson at quarterback.

By the time Walsh became the San Francisco head coach in 1979, the NFL had tipped the rules in favor of the passing game. Offensive lineman could now use their hands when blocking, which made it easier to protect the quarterback. Defensive backs could now put their hands on a receiver only within 5 yards of the line of scrimmage instead of everywhere on the field.

Walsh thrived in San Francisco during the 1980’s. His teams won three Super Bowls based on his high powered aerial attack. With this success, the rest of the NFL began to adopt the West Coast offense and throw the ball more.

The fraction of pass plays rose drastically after 1978 and then continued to rise to 55% by the end of the 1980’s. This infatuation with the pass offense has only accelerated since the turn of the century. By 2016, the fraction of pass plays has reached almost 60%, as the NFL no longer has many blocking tight ends or full backs.

With all of these pass plays, one might expect the efficiency of these plays to decrease. The defense begins to take away the passing game, making it easier for the offense to run the ball.

However, the exact opposite has happened. Yards per pass attempt, a number that includes the negative yardage from sacks, increased drastically after the 1978 rule changes. This pass efficiency has continued to increase after the turn of the century despite an increasing fraction of pass plays.

In contrast, the NFL run game has seen only small gains in efficiency. While the NFL averaged about 4 yards per carry in the 1980s, ball carriers now squeak out 4.18 yards per carry from 2012 through 2016.

Not only has the pass game become more efficient, it has also become safer. In the 1970’s, more than 5% of pass attempts ended in the hands of the defense. But with the adoption of the short passing game, the interception rate has dropped in a linear fashion and reached a new low of 2.2% in 2016.

The passing game is the gift that keeps on giving in the NFL. The increases in efficiency and safety despite an increasing fraction of pass plays suggest that the pass game will continue to dominate the NFL.

So let’s take this idea to its logical extreme. Should NFL teams throw on every down?

The NFL’s most efficient play

To understand whether teams should throw on every down, let’s ask a question first proposed by Grantland writer Robert Mays in 2013: what is the NFL’s most efficient play?

To determine the answer, Mays consulted the ESPN analytics group. They perform calculations that determine the value of each play through the idea of expected points added (EPA). This concept puts the result of a play in context; a 2 yard gain means more on 3rd and 1 than 1st and 10.

To understand EPA, suppose a team has a 1st and 10 at their own 20 yard line. They could drive the length of the field for a touchdown for +7 points or kick a field goal for +3 points. In the worst case, an interception gets returned for a touchdown, netting -7 points for the offense.

Expected points added (EPA) is the expected points gained or lost from a play. For example, a team starts with 1st and 10 at its own 20 yard line, which has an expected points of 0.3. Let’s say they gain 20 yards on first down. The resulting 1st and 10 from the 40 has an expected points of 1.3. Hence, the 20 yard gain results in a EPA of 1.0.

Mays had the ESPN analytics group look at the EPA for different types of plays. As a baseline, they first looked at rush plays based on data from the 2009 through 2013 seasons. On average, a rush play earns -0.04 EPA.

In case you missed the negative sign in front of that number, the average run play in the NFL loses points for the offense. That might be the biggest indictment of running the ball in the NFL yet.

First, the efficiency of run game by yards per carry has stagnated since the late 1970’s. With a closer inspection of run plays based on EPA, the average run play hurts a team in its quest to score more points than the other team. During the 5 year period of the study, only 4 of 32 NFL teams averaged a positive EPA on run plays.

In contrast, the average pass played earned a team +0.04 EPA. Throwing the ball moves a team in the right direction to score more points than the opponent.

A certain type of pass play does much better than the typical. The play action pass nets the offense +0.17 EPA. The deception of play action, a play in which the quarterback fakes a hand off before throwing down field, leads to a staggering increase in the point value of the play.

Should a team throw the ball on every play? No, since the lack of run plays kills the deception necessary for play action. Throwing on every down makes the offense predictable.

However, the study also found that a team doesn’t need to run the ball effectively to make play action work. Remember those 4 teams that had a positive EPA on run plays over the 5 seasons of the study? Only one of them had a top 10 EPA for play action. This team (New England) had a quarterback named Tom Brady.

During this period, Adrian Peterson of the Minnesota Vikings was the most dominant running back in the NFL. However, Minnesota never excelled at play action. The Vikings ranked 21st in EPA on play action. In 2013, Buffalo led the NFL in rush attempts but ranked last in EPA on play action.

The analytics suggest that a team doesn’t have to run the ball well to excel at play action. Despite the inherent deception, play action is a pass play whose success relies on a accurate pass down the field. A team needs a good quarterback to make that pass. In the study, the top play action teams (New England, Green Bay, Pittsburgh) had elite quarterbacks (Tom Brady, Aaron Rodgers, Ben Roethlisberger, respectively).

Even in showing that teams need to run the ball, we run into the insignificance of running the ball well.

How to evaluate NFL statistics

In Super Bowl XXV, Bill Belichick’s plan to let Thurman Thomas rush for 100 yards worked, maybe too well. Against a small defense designed to slow down the pass, Thomas ran for 135 yards on 15 carries, a staggering 9 yards per carry. In the second half, he broke off a 31 yard run for a touchdown.

The game ended when Bills kicker Scott Norwood sent a field goal attempt wide right. The Giants won the Super Bowl 20-19.

The Giants did not win the game solely because of Belichick’s defensive plan. The offense generated two long scoring drives in the second half that took time off the clock. And I would bet my life savings Belichick did not want his defense to allow that 31 yard touchdown run to Thomas.

But, as Halberstam discusses in Education of a Coach, Belichick did want the Bills to pick up small gains on the ground if it meant keeping Jim Kelly from throwing the ball. He understood that rushing means little to winning in the NFL.

Each year, the significance of the passing game in the NFL becomes more apparent. Teams offer huge contracts to cornerbacks like Patrick Peterson while future Hall of Fame running backs like Adrian Peterson wait for a free agent deal.

Teams also now look to the NFL draft to fill needs in the running game. They can draft a running back good enough to crank out 4 yards per carry behind a decent offensive line. He doesn’t cost much on a rookie contract, leaving salary cap resources for the pass game on offense and defense.

The insignificance of the run game should also impact NFL handicapping. You have limited time to evaluate injuries, so focus on those that affect the passing game. The quarterback is an obvious adjustment, and an injury to a top receiver matters too. But cornerbacks do not get the same attention even though they might be the NFL’s most difficult players to replace.

Passing dominates the NFL. Rushing hardly matters.

Filed Under: Football Analytics, Football Passing Analytics, National Football League

How to make accurate football predictions with linear regression

By 6 Comments

As a smart football fan, you would like to identify overrated college football teams. This is a difficult task, as half of the top 5 teams in the preseason AP poll have made the College Football Playoff the past 4 seasons.

However, analytics has a nifty trick for identifying teams that don’t belong near the top of the preseason polls.

In addition, this trick lets you look at the statistics on any major media site and identify teams playing above their skill level. In a similar fashion, you can find teams that are better than their record.

The trick relies on regression to the mean.

When you hear the word regression, you probably think of how extreme performance during an earlier period most likely gets closer to average during a later period. It’s difficult to sustain an outlier performance.

This intuitive idea of reversion to the mean is based on linear regression, a simple yet powerful data science method. It powers my preseason college football model that has predicted almost 70% of game winners the past 3 seasons.

The regression model also powers my preseason analysis over on SB Nation.  In the past 3 years, I haven’t been wrong about any of 9 overrated teams (7 correct, 2 pushes).

Linear regression might seem scary, as quants throw around terms like “R squared value,” not the most interesting conversation at cocktail parties.  However, you can understand linear regression through pictures.

Let me explain.

1. The 4 minute data scientist

To understand the basics behind regression, consider a simple question: how does a quantity measured during an earlier period predict the same quantity measured during a later period?

In football, this quantity could measure team strength, the holy grail for computer team rankings. It could also be turnover margin or win percentage in one score games.

Again, consider this question:

How does a quantity in an earlier period predict the same quantity during a later period?

Some quantities persist from the early to later period, which makes a prediction possible. For other quantities, measurements during the earlier period have no relationship to the later period. You might as well guess the mean, which corresponds to our intuitive idea of regression.

To show this in pictures, let’s look at 3 data points from a football example. I plot the quantity during the 2016 season on the x-axis, while the quantity during the 2017 season appears as the y value.

If the quantity during the earlier period were a perfect predictor of the later period, the data points would lie along a line.  The visual shows the diagonal line along which x and y values are equal.  

In this example, the points do not line up along the diagonal line or any other line.  There is an error in predicting the 2017 quantity by guessing the 2016 value.  This error is the distance of the vertical line from a data point to the diagonal line.

For the error, it should not matter whether the point lies above or below the line.  It makes sense to multiply the error by itself, or take the square of the error.  This square is always a positive number, and its value is the area of the blue boxes in this next picture.

The area of the blue boxes is the mean squared error.

In the previous example, we looked at the mean squared error for guessing the early period as the perfect predictor of the later period.  Now let’s look at the opposite extreme: the early period has zero predictive ability.  For each data point, the later period is predicted by the mean of all values in the later period.

This prediction corresponds to a horizontal line with the y value at the mean.  This visual shows the prediction, and the blue boxes correspond to the mean squared error.

The area of these boxes is a visual representation of the variance of the y values of the data points.  Also, this horizontal line with its y value at the mean gives the minimum area of the boxes. You can show that every other choice of horizontal line would give three boxes with a larger total area.

Regression requires finding the line that minimizes the squared error, or the area of the boxes.  This line is called the best fit line, and the next visual shows the best fit line along with the corresponding minimum mean squared error.

In trying to scare off normal people, the quants will thumb up their nose and say things like “the best fit line explains 70% of the variance.”  Even worse, they might call this the “R squared” value.

You can understand this statement through the pictures.  The best fit line explains 70% of the variance means that the total area of the red boxes is 70% less than the original blue boxes of the horizontal line.

In this example, the best fit line causes a significant reduction in the area of the box corresponding to the left data point.  The box gets larger for the middle point (the blue box is obscured by the red box corresponding to the best fit line). But overall, the area of the red boxes are 70% less than the blue boxes.

You can also think about the error the best fit line doesn’t explain.  The area of the red boxes is 30% of the area of the blue boxes.  This 30% is the remaining variance after the best fit line removes 70% of the original variance.

The higher the R squared value (70% in the example above), the smaller the red boxes of the best fit line.  The line explains the data very well.

In contrast, the lower the R squared value, the larger the red boxes of the best fit line, which will be more horizontal.  It doesn’t do much better than the horizontal line of the average.

Get my best football predictions

The Power Rank specializes in making accurate football and March Madness predictions.

To get a sample of football predictions usually reserved for paying members of the site, enter your best email and click on “Sign up now!”









 

 

 

 

 

 

 
 

2. Persistence versus regression to the mean

The data above come from my team ratings in college football.  To develop these numbers, I take margin of victory in games over a season and adjust for strength of schedule through my ranking algorithm.  The rating gives an expected margin of victory against an average team on a neutral site.

For the 2014 through 2016 season, here is how the team rating for the season predicted the next season.

The data hug the best fit line, and ratings from the previous season explain more than half of the variance in ratings the following season (54.1%).

Compare this to the same plot with turnover margin, or take aways minus give aways.

For turnover margin, the best fit line is almost flat.  Turnover margin in one season explains 2.6% of the variance in turnover margin the following season.

From these two plots, we can make two statements:

  1. Team strength in college football as measured by adjusted margin of victory tends to persist from year to year.  This will be useful in making a preseason college football model.
  2. Turnover margin regresses to a mean of zero from year to year.  This implies that turnover margin last season has almost zero ability to predict turnover margin this season.

When most people talk about regression, they usually mean the strong type we see in turnover margin.  To see this from a different perspective, let’s consider the wins and losses of Coach Average.  His results come from flipping of a coin with an equal chance for a win and loss.

For this visual, I generated this data once with 8 lines of Python code.  In no way did I search for a sequence with streaks. However, streaks almost always appear in the sequence.

In this simple experiment, the flipping of any one coin has no impact on the outcome of the next coin.  The code makes each game for Coach Average independent of all other games.

Regression to the mean implies that despite a hot 8-2 start for Coach Average, he should still expect to win half of the next 10 games.  In fact, he wins 6 of the next 10 games.

Coach Average also expects to regress to .500 after 9 straight losses starting on game 19.  However, he loses 6 of the next 8 games. 

3. Skill versus luck

Phenomena in the real world are not as simple as this coin flipping experiment, and we need to be cautious in making statements about sports. When a quantity like turnover margin has no ability to predict future turnover margin, it doesn’t imply a lack of skill in preventing or forcing turnovers.

While the analytics community doesn’t have a complete picture of turnovers, a few key insights have started to emerge.  The game situations matters in turnovers. Over the past decade in college football, teams in the lead have committed a third of all turnovers. For teams ahead by a touchdown or more, the rate drops to 14% of all turnovers.  

This would help explain why Alabama has posted +68 turnover margin the past 8 seasons.  The Crimson Tide failed to have a positive turnover margin only in 2014, when they gave away the ball 2 more times than they took it away.

The dependence of turnovers on game situation makes sense.  Teams in the lead tend to run the ball, especially later in the game.  Turnovers happen at a lower rate on running than passing plays.  If a team faces a deficit, they need to throw the ball to get back into the game.

For another example of how regression doesn’t necessarily imply a lack of skill, let’s turn to college basketball.  Ken Pomeroy wondered how much control teams have over three point shots. He asked how a team’s 3 point shooting percentage in the first half of the conference season predicted the second half.

The visual shows his results.


The left panel shows 3 point percentage on offense.  The first half of the conference season has almost no ability to predict the same quantity later in the season.  Does that mean shooting is not a skill? Tell that to Steph Curry.

The visual also shows the strong regression for 3 point field goal defense. This suggests a lack of skill in defending the 3.  To confirm this, Pomeroy performed a more detailed study on 3 point defense for teams over a 5 year period.  He concluded there is some skill, but randomness plays a bigger role than anyone expects.

When a quantity regresses to the mean like turnover margin or 3 point shooting percentage, it doesn’t necessarily imply a lack a skill.  Fumbles may regress to the mean, but that doesn’t mean a running back isn’t fumble prone if he palms the ball with one hand while running through the line of scrimmage.

However, it is safe to say that randomness plays a large role in turnovers and 3 point shots.

4. How to make preseason college football predictions

USC had high expectations heading into 2017.  Sam Darnold took over the starting QB job the previous season and led the Trojans to a 9-0 finish, which included a dramatic win over Penn State in the Rose Bowl.

At the start of the 2017 season, the pollsters put USC 4th in the preseason poll (both AP and Coaches).  This made Clay Helton’s team the favorite to make the College Football Playoff out of the Pac-12 conference.

In contrast, no one knew quite what to expect from Georgia.  Just like USC, 2017 was their second under a young coach, Kirby Smart.  But in contrast to USC, they struggled in 2016.

Georgia started true freshman Jacob Eason at quarterback, who delivered a mediocre 55% completion rate.  They ranked 81st in my adjusted yards per attempt.

Georgia went 8-5 in 2016, a record acceptable only for new coaches in Athens.  To start 2017, they landed at 15th in the preseason AP poll.

So what does regression say about these two teams?  Each year, I put together a preseason college football model that uses regression on many variables.

In college football, team performance tends to persist from year to year. Programs like Alabama have financial resources and traditions that Rice will never have.  These teams will not swap places in the college football hierarchy.

Hence, my preseason model uses the past 4 years of team performance to predict the next season.  This part of the model says that team is most likely to perform as some combination of their last 4 years, with recent years weighted more.  This makes the model cautious about an outlier season or 9 games.

The preseason model also considers turnover margin.  Turnovers can impact the scoreboard, as a key fumble halts a game winning drive, or an interception returned for a touchdown turns a tight game into a laugher.

However, turnover margin regresses to the mean of zero from year to year in college football.  Hence, the model uses turnover margin in each of the past 4 seasons. This holds back the excitement for a team that made a huge jump in rating with a +25 in turnover margin.

Last, the model considers returning starters.  More experience implies better performance for college football teams.

Over the past 3 season (2015-17), this regression model for college football has predicted 69.8% of game winners.  This rate doesn’t include easier to predict cupcake games with FBS teams facing inferior FCS teams.  The model only makes predictions for games with two FBS teams.  

Heading into 2017, the preseason college football model had USC 16th.  In the previous 4 seasons, USC had never finished the season higher than 14th in my college football rankings.  Despite their impressive 9-0 finish in 2016, they only rose to 14th because of a poor start.

The model doesn’t distinguish between returning starters at different positions.  The quarterback has an outsized impact on a football team, and Darnold’s status as a top NFL prospect could convince you of USC as higher than 16th.  However, 4th in the AP poll seemed too optimistic.

In contrast, the regression model agreed with the AP poll on Georgia.  The model had the Bulldogs at 18th while the polls had them at 15th.  Analytics and polls agreed on Georgia as a solid top 25 team but not a playoff contender.

During the actual 2017 season, USC did not live up to their top 4 ranking.  They dropped an early road game at Washington State. Then their playoff hopes ended when Notre Dame stomped their defense for 8.4 yards per carry in a 49-14 win.

Georgia’s season got off to a distressing start as Eason sustained an injury in the first game.  However, this turned into a blessing, as true freshman Jake Fromm took over and had a brilliant season. Georgia’s pass offense finished 10th in my adjusted yards per attempt.

The defense also improved, as they jumped from a solid 28th in my adjusted yards per play in 2016 to an elite 3rd in 2017.  This unit had only one bad game when they allowed 40 points at Auburn. However, they atoned for this blip by holding Auburn to 7 points in the SEC title game win a few weeks later.

Georgia made the College Football Playoff, and they took Alabama to overtime in the championship game before losing.

Here’s the take home message about college football preseason predictions: It’s much easier to predict regression for a team like USC than a sudden rise for Georgia.  

In each of the past 3 seasons, I’ve written about 3 overrated college football teams in the preseason polls on Football Study Hall, an SB Nation site (2015, 2016, 2017).  This analysis combines my regression model with knowledge of programs.

Looking back on these predictions, I’ve been right about 7 of 9 teams.  The two teams, Penn State and Oklahoma State, finished lower in the final poll than the preseason poll in 2017.  However, I’ll call these predictions a push as they both performed better than I predicted.

There’s a good reason I have never written a corresponding 3 underrated teams article.  A regression model is unlikely to identify breakout teams like Georgia in 2017.  Their true freshman quarterback worked out, and their defense made a leap.  

There is a random element to teams that make a sudden rise.  These predictions are more difficult than finding an overrated team by a regression model.

5. How to make accurate predictions with regression

I want you to take home two main points.

First, quantities in football like turnover margin show very little persistence from an earlier period to a later period.  These quantities regress to the mean, as your best guess for the later period is the average.

To find football teams that might not be as strong as their record suggests, look for teams with large, positive turnover margin.  In contrast, teams with large, negative turnover margin might be better than their record. Use these links for data on college football and the NFL.

As another example, consider an NFL team’s record in close games from year to year.  Based on data from the 2012 through 2017 season, the visual shows that one year explains 1.3% of the variance the next year.

A team like Oakland, who went 9-2 in one score games in 2016, should not expect the same good fortune again.  The Raiders went 4-3 in one score games in 2017. However, regression to the mean isn’t a perfect predictor, as Cleveland has a 2-16 record in one score games the past 3 seasons.

Second, some quantities like team strength in college football tends to persist from year to year.  This allows for predictive models based on linear regression.

Even with this persistence, the models still predict regression for outlier performances, both good and poor.  The 9-0 stretch for USC to end 2016 serves as an example. However, regression models can not predict teams that jump from ordinary to the outlier, like Georgia in 2017.

These ideas apply for both my preseason regression model at The Power Rank and Bill Connelly’s S&P+ numbers.  Use these rankings as a guide to find overrated teams near the top of the polls.

Filed Under: College Football, Football Analytics, National Football League

The Football Analytics Resource Guide

By 1 Comment

With less than 4 weeks before 2018 football season, you’d like get up to speed on football analytics. You need the insight that makes you the smartest fan on your block or message board.

What do the numbers say about skill versus luck in various aspects of football? How can coaches make better decisions during games?

To help you on your journey, I’ve complied the 9 best articles on football analytics. Instead of searching the web for resources, check out my curated list, which will show you the following:

  • the huge impact plays that teams have little control over (no, it’s not turnovers)
  • the simple decision that increases a team’s win probability that no NFL coaches make
  • the paradoxical reason that luck plays an increasing factor in NFL quarterback performance
  • the mistake that almost all college football statistics sites make
  • why Belichick was correct in going for it on 4th and 2 from his own 28 yard line against the Colts

For each resource, I summarize the essential aspects and then provide a link for more details. It’s a long article, but you can consume the 9 parts independently.

To check out The Football Analytics Resource Guide, click here.

Filed Under: Football Analytics

How Philadelphia used analytics to win the Super Bowl

By 2 Comments

After beating New England in the Super Bowl, Philadelphia Eagles head coach Doug Pederson joined the Game Theory and Money podcast to talk analytics. I’ve transcribed his words here, with edits to make it more readable.

First, Pederson in general terms how he views analytics.

Our team has a lot of information and data that pertains to situational football. What’s amazing is that I do a study, and boom, I have 5 to 10 years of data. However much data I want to pull, I can get a relatively quick and accurate answer with some of the data.

And for me, it’s about deciphering what I want to use. What helps us win football games, and what helps us win the Super Bowl, when you’re going against a tremendous opponent in the New England Patriots. What advantages can I have through the numbers, whether it’s 4th downs, two point conversions, or whatever. It might even be on a player.

I can use that information in formulating game plans and making good sound decisions on game day.

The most interesting comment Pederson made was about how a fourth down decision affects win probability, a topic Brian Burke studied long ago.

I think where the numbers and analytics comes in is by field position, by down and distance, and win probability of the success rate of getting that fourth down.

I’m constantly being communicated with. I have a coach upstairs that’s giving me this information real time, game day. Hey coach, if we can get it to 4th and 2, 4th and 3, this is the success rate here in this situation.

Now ultimately, it’s my decision whether to go for it in those situations. But as our numbers have shown, I’ve elected to go for it on 4th down more than most.

The decision to go for it on 4th down isn’t only about analytics either:

Well, people say I’m an aggressive player caller. But there’s a lot of information that’s been studied prior to that decision, and a lot of these decisions are made before the game even starts.

Like the other night in the Super Bowl, I’m faced with that 4th and 1 at the minus 45 yard line. I elected to go for it with 5 minutes to go in the game. That to me was more about trusting your player than any analytic number.

Even though it was favorable to go for it, I trusted our guys in that situation, and I think it’s important to have that type of trust with your players.

He’s talking about going for it on 4th and 1 from their 45 yard line with 5:39 to go in the game. The Eagles converted a short pass from Nick Foles to Zach Ertz, and this drive ended with the go ahead touchdown that won the game.

I may be cherry picking the good situations in which analytics have worked out for an underdog like Philadelphia. However, the other big story in football analytics is how Cleveland’s experiment with numbers ended in a 0-16 record this season.

Philadelphia may ride Carson Wentz and numbers to multiple Super Bowl titles. Or Doug Pederson might get fired after the 2019 season. Either way, it’s important to take this snapshot of analytics at the end of the 2017 season.

You can listen to the full interview with Doug Pederson from 23:18 to 33:25 of this episode of Game Theory and Money.

Filed Under: Doug Pederson, Football Analytics, National Football League

Podcast: John Urschel, NFL lineman, on football analytics and math research

By 1 Comment

On this episode of the Football Analytics Show, John Urschel, lineman for the Baltimore Ravens and Ph.D. candidate in mathematics at MIT, joins me for a wide ranging conversation. Topics include:

  • The NFL’s progress in using video tracking data
  • The football analytics problem John is working on
  • The acoustic operator in those Bose commercials with J.J. Watt
  • The breadth of math research in which John is engaged
  • The hidden factor in his success as an NFL lineman

John is an amazing human being, and I hope you enjoy this episode.

You also might be interested in an interview I did with John 3 years ago before he got drafted by the NFL.

To listen on iTunes, click here.

To listen here, click on the triangle.

Filed Under: Football Analytics, John Urschel, Podcast
Next Page »