How Andy Reid wins football games with interceptions

Andy Reid’s teams throw a low rate of interceptions. The visual shows how his teams in Philadelphia and Kansas City have had a lower than NFL average interception rate (interceptions divided by attempts) in all but 3 of 18 seasons.

Reid’s Eagles had a particularly good stretch of pick suppression from 2000 through 2004. Led by quarterback Donovan McNabb, Philadelphia never won fewer than 11 games in any of those 5 seasons.

Despite a decreasing interception rate across the NFL, Reid has continued to beat the NFL average over the past four years in Kansas City. Led by quarterback Alex Smith, the Chiefs have won 43 regular season games and never dipped below 9 wins in any one season.

The randomness of turnovers

The visual goes against the typical quant narrative that turnovers are random.

For example, I’ve shown this visual that shows the relation between interception rate the first 6 games of the college football season versus the remainder of the season.

The lack of correlation between these quantities shows that you can’t predict a team’s interception rate later in the season based on the same quantity during first 6 games.

This suggests interceptions are random, and a team has a 50% chance to have a better or worse than average interception rate. However, if you assume this for Andy Reid’s teams, there’s only a 0.37% chance his teams would have had 3 or fewer seasons with a below average interception rate.

Randomness certainly plays a role in interceptions. No one who has ever seen a tipped pass fall into the hands of a defender should doubt that.

However, the Reid visual suggests that some coaches can suppress interceptions over a very large sample of games.

Steelers at Chiefs

This has implications in predicting the outcome of the Steelers at the Chiefs playoff game this weekend.

Kansas City doesn’t seem like much of a Super Bowl threat with the 16th and 11th ranked pass offense and defense, respectively, by my adjusted yards per attempt. I use these pass efficiency numbers to evaluate teams for two reasons:

  • My research shows the importance of passing over rushing in the NFL.
  • Turnovers have little impact on yards per pass attempt.

However, if Kansas City is truly skilled at not throwing interceptions, then these pass efficiency numbers will underestimate their team strength.

Team rankings based on adjusted margin of victory might be a better way to evaluate Kansas City. Their low interception rate will impact margin of victory, as I’ve found that an interception is worth 5 points in the NFL.

My member numbers combine both pass efficiency and margin of victory to make Kansas City a 1.3 point favorite against Pittsburgh. However, my team rankings based on only points would make Kansas City a 3 point favorite.

I interviewed Ben Alamar on the Football Analytics Show this week, and his FPI (Football Power Index) makes the Chiefs nearly a 5 point favorite. They use an expected points added, a metric which accounts for interceptions but their own twist. To listen to that part of the discussion, go to 14:50 of my interview with Ben Alamar.

How often Jim Harbaugh’s 4th down decisions agree with analytics

This is a guest post from Tony Kaminski, a recent University of Michigan graduate and engineer. You can follow him on Twitter or check out his site Big House Analytics.

How does Jim Harbaugh’s fourth down decisions stack up to analytics?

The numbers say that coaches should go for it on fourth down way more than they do. Berkeley professor David Romer started this research back in 2006, and Brian Burke, now with ESPN, has performed the most complete study based on NFL data.

My interest in this topic as a Michigan fan began with a conversation with Craig Ross, a member of WTKA MGoBlog Roundtable and author of two books on Michigan sports. In his book The Search for the Unified Field Theory (Football Version), he writes

So here’s the deal. Einstein’s follies aside, I believe there is a unified field theory in football, a fundamental set of ideas or equations that can help to explain “what happened” in a football game, why the game was won or lost. More than this, I think this theory allows a coach to make choices and structure his team and program in a way that squeezes out a few more wins than might otherwise be the case.

And therein lies our aim: finding this unified theory. Craig and others have good reason to believe that not punting very much, or at least substantially less than coaches tend to do currently, is a part of applying this theory to winning more games.

The natural starting point of this conversation is the work of David Romer, the Berkeley economist. An excerpt from a New York Times profile of Romer explains the motivation looking at 4th downs:

Romer, a lifelong sports fan who is a professor at the University of California, Berkeley, came up with the idea to rigorously examine fourth-down plays after listening to a radio broadcast of an Oakland Raiders game in his car about a decade ago. Although the Raiders had the ball in striking distance of the end zone, one of the commentators remarked that they would be smarter to kick a near-certain field goal rather risk going for a touchdown.

“I am pretty analytic,” Romer recalled telling himself. “That is a pretty shallow way of thinking about it.”

So, after stewing over the idea for a couple of years, he set out to tackle the great fourth-down debate.

Romer’s work has been largely met with skepticism in coaching circles because the difficulty in overcoming conventional wisdom. Coaches at big-time universities or professional franchises with nine-figure payrolls often stick to the proverbial book — and why not?

If you fail underneath the brightest lights, it’s easier to defend yourself when your decisions echo the conventional wisdom. Wins come at a premium, and no career has less job security or more turnover than football coach. So you do everything you can to hold onto that gig, because they don’t come around often.

But doing great things, in the private sector or on the football field, involves bucking convention. Romer has shown this means punting less. Way less.

You can read this article for a quick tutorial on Romer’s work and its application. While the New York Times model doesn’t explicitly reference the professor, their “4th Down Bot” comes to a very similar conclusion, just with more data to back it up. It boils down to this:

Especially near the middle of the field, coaches need to be more aggressive. In this article, we’ll look at Jim Harbaugh’s fourth down decisions since his arrival at Michigan. A few points on this analysis.

  • While Romer and the New York Times used NFL data, Romer believes the findings also apply to the college game. (Craig recounts an email conversation that him and the professor had in his book where Romer explains why.)
  • Bill Connelly’s S&P has designated garbage time as being up by 25 points or more in the second quarter, 22 or more in the third, or 17 or more in the fourth. For the 2015 analysis, I filtered this out arbitrarily, in a more liberal manner than Bill does, based on my recollections of the individual games. In 2016, I followed Connelly’s rules.
  • This is not an indictment of Harbaugh’s decision making on fourth down, merely an analysis of how closely his decisions echo Romer’s advice. There are specific team, opponent, or situation-variant times where it would be best to not follow the chart above.
  • This data does not reflect the final outcome of the decision made but only the decision to go for a fourth-down conversion or punt itself.

Let’s first look at Harbaugh’s 2015 fourth down decisions in these tables. The color green designates a decision that follows what analytics recommends. Yellow means it did not but was defensible given the score, situation, or general trend of the game (more on that later). Red designates a time when Harbaugh probably should have gone for it on fourth down opposed to punting.

Let’s discuss a few of the decisions marked yellow, or ones I found defensible given the game situation. (Maybe I was a bit too friendly to our coach, I’m a fan and proud alum, after all — leave a comment if you disagree with any of my designations, and we’ll continue the conversation).

For instance, Michigan punts on fourth and seven from the Michigan State 38 with 1:12 to go in the first half of a 10-7 game. That’s not technically the correct decision but consider the following:

  • Michigan State is out of timeouts
  • Michigan had allowed exactly seven points over the prior 17 quarters
  • Michigan had one of the best punters in college football in 2015

Michigan State runs three quick plays without scoring before halftime.

Against Northwestern, it’s fourth and nine from their 34 with 4:19 to go in the second quarter. Michigan has a 21-0 lead. The chart says to try a field goal but that’s too deep for then-freshman kicker Kenny Allen, who had missed a 44-yard attempt earlier in the season. Michigan punts, and then Jourdan Lewis houses an interception on the ensuing possession that erases any doubt about the outcome.

This isn’t to say I agree with all of the times Harbaugh deviated from the research. I was upset by the decision to punt from Ohio State’s 36 on fourth and five less than five minutes into that game. Given the talent discrepancy between the Wolverines and Buckeyes, who littered the first round of the NFL draft (literally, probably), points would be at a premium, and we passed up an opportunity to get some early.

Ultimately, Harbaugh had a 77.8% “success rate” — out of Michigan’s 54 fourth down decisions, Harbaugh went with analytics or I found the punt defensible on 42 of them. If you want to go by the book (meaning the punts I deemed as defensible but not Romer-approved counting against our success rate instead of for it), our score is 59.3% (32 of 54).

The following table shows my results for the 2016 season through the Ohio State game.

On the following fourth downs, Harbaugh went for the first down.

This analysis shows a 91.3% success rate (42 of 46). The results were so staggering that I had to have a friend double check my work. If you went purely by the numbers, our success rate is 76.0% (35 of 46) . By either metric, this is a substantial improvement from last season.

A couple notes:

  • Michigan blew out seven of their 12 regular-season opponents, so this chart was significantly easier to compile than last year’s.
  • I did not chart the “defeat with dignity” portion of the Michigan State game, the final offensive play against Indiana (it was a full-blown blizzard in Ann Arbor at that point and IU had effectively conceded), or the fourth-down attempt in overtime against Ohio State since we obviously didn’t have a choice.
  • There were only four punts I gave the “red” designation, and even then, I felt that I may have been being unreasonably harsh on Harbaugh, since these were coming up correct at a much higher clip than last season.
  • Michigan was 12 for 18 on fourth-down conversion attempts in 2016 compared to six for 16 in 2015. More of these came in garbage time this year than last, hence the disparity in charted attempts.

Harbaugh’s decision making already agreed with analytics a good bit in 2015, but 2016 represents an even bigger step forward. I have a few hypotheses on why.

First, Michigan has a larger analytics team than under previous regimes. It’s not hard to believe that they had a part in this trend.

Second, Harbaugh had a different situation on defense these two years. In 2015, the defensive production fell off with the injury of Ryan Glasgow, most notably when Indiana gashed Michigan for 307 rushing yards in Bloomington and the dreadful Ohio State game.

This was not the case in 2016. Don Brown’s defense was stout all season long, which gave Harbaugh the confidence the defense could get a stop on a short field.

Does Aaron Rodgers draw more pass interference penalties?

On my recent appearance on Beating the Book, we were discussing Aaron Rodgers and why he wasn’t playing as well. Host Gill Alexander thew out the idea that Rodgers’ performance might not seem as bad if we included drawn pass interference penalties.

I dug into the 2016 play by play data through week 10 to find out. Rodgers has drawn 7 pass interference penalties, just above the team average of 6.

Drawing pass interference penalties doesn’t seem like a skill, as Drew Brees has 3 while Blake Bortles and Ryan Fitzpatrick have 12 and 11 respectively.

Rodgers does seem to draw pass interference penalties deep down the field. Here are the yardage gains on these penalties: 44, 18, 30, 40, 13, 28, 66. If you include these plays, it would help a pass offense that has averaged 5.8 yards per attempt, 26th in the NFL.

Here are the full results for defensive pass interference penalties for all teams during the first 10 weeks of the 2016 season.

  • Arizona, 7.
  • Atlanta, 6.
  • Baltimore, 6.
  • Buffalo, 4.
  • Carolina, 6.
  • Chicago, 4.
  • Cincinnati, 9.
  • Cleveland, 6.
  • Dallas, 2.
  • Denver, 9.
  • Detroit, 9.
  • Green Bay, 7.
  • Houston, 6.
  • Indianapolis, 5.
  • Jacksonville, 12.
  • Kansas City, 1.
  • Los Angeles, 5.
  • Miami, 2.
  • Minnesota, 4.
  • New England, 5.
  • New Orleans, 3.
  • New York Giants, 7.
  • New York Jets, 11.
  • Oakland, 12.
  • Philadelphia, 3.
  • Pittsburgh, 5.
  • San Diego, 8.
  • San Francisco, 3.
  • Seattle, 6.
  • Tampa Bay, 7.
  • Tennessee, 9.
  • Washington, 6.

Finally!! NFL preseason rankings based on wisdom of crowds

nfl2016_preseasonYou want to know the strength of your NFL team. You’ll take any analytics that can sort through the preseason noise of the NFL.

In college football, team strength tends to persist from year to year. This makes it possible to use previous seasons to predict the current season.

However, looking at past years does not work in the NFL since team performance regresses to the mean. The salary caps levels the playing field for all 32 teams. Injuries and luck can derail teams with the highest expectations.

However, we can use a different trick from college sports to rank NFL teams in the preseason. Let me explain.

Wisdom of many sports writers

Preseason polls in college sports are remarkable predictors of success.

In the preseason AP poll, the higher ranked team has won 60.2% of bowl games that season since 2005 (174-115 with no prediction in 91 games). The preseason Coaches poll also performs well at a 60.9% rate (182-120 with no prediction in 73 games).

The combined wisdom of sports writers or coaches lead to remarkable rankings. However, the accuracy of these polls decrease once the season starts. The writers or coaches tend to react too strongly to wins and losses. By the end of the season, the higher ranked team in the AP polls wins 56.6% of bowl games.

However, the AP poll is a remarkable tool before the season starts. Let’s created the same type poll for the NFL.

Ensemble NFL preseason rankings

Every major sports media site publishes preseason power rankings. I aggregate these rankings from over 20 sites.

To make predictions, each teams also needs a rating, or an expected margin of victory against an average NFL team. I use historical results from the last 10 years of my NFL team rankings, which take margin of victory and adjust for strength of schedule.

I’ve done this calculation for the last 3 years, and the model has predicted the winner in 64.5% of games (515-284 which doesn’t count two tie games). The opening Vegas line gets 66% of games correct on average.

Preseason rankings for 2016

In these wisdom of crowds rankings for 2016, it seems like most gave the Patriots a rank with Tom Brady, who won’t play the first four games of the season.

1. Carolina, 7.6
2. Seattle, 7.2
3. Arizona, 6.7
4. New England, 6.2
5. Green Bay, 5.9
6. Pittsburgh, 5.4
7. Denver, 4.3
8. Cincinnati, 4.2
9. Kansas City, 3.3
10. Minnesota, 1.6
11. Oakland, 1.2
12. Houston, 1.0
13. New York Jets, 0.9
14. Washington, 0.7
15. Baltimore, -0.3
16. Indianapolis, -0.4
17. New York Giants, -0.6
18. Dallas, -0.6
19. Jacksonville, -1.4
20. Buffalo, -1.4
21. Tampa Bay, -1.9
22. Atlanta, -2.3
23. Detroit, -2.4
24. Miami, -3.1
25. New Orleans, -3.4
26. Los Angeles, -4.1
27. Chicago, -4.2
28. San Diego, -4.6
29. Philadelphia, -4.7
30. Tennessee, -5.5
31. San Francisco, -7.9
32. Cleveland, -8.1

The predictions for week 1 are posted on the predictions page.

How computer rankings make you smarter about football

Predicting Super Bowl in 2016 was the ultimate test between eyes and numbers.

By the eye test, Carolina looked like the clear favorite over Denver. The Panthers had a stellar 17-1 record, and they destroyed two of the NFL’s best teams, Seattle and Arizona, to make the Super Bowl.

The eye test also favored Carolina at the quarterback position. Cam Newton had an Most Valuable Player caliber season, a touchdown machine at the pinnacle of his game.

However, the eye test for Super Bowl 50 didn’t hold up. In the first quarter, Von Miller stripped Cam Newton of the ball. Denver recovered for a touchdown that gave them a 10-0.

Carolina’s offense never left the gate to take off for flight. Despite an anemic offense, Denver won 24-10 with the help of a few critical turnovers.

In contrast to the eye test, numbers suggested Denver wasn’t as overmatched as they seemed against Carolina in Super Bowl 50. This insight was based on computer rankings and their adjustments for strength of schedule.

Let me explain.

Margin of victory

It should be obvious that a team ranking system should consider margin of victory in games.

Do you care that Amazon has lower prices than your neighborhood book store? No. It’s the 40% discount on all titles that compels you to buy online.

The same lesson applies to computer rankings.

The Power Rank’s team rankings start with margin of victory in games. However, this raw metric didn’t tell the entire story about Carolina. The Panthers had an average margin of victory of almost 13 points, by far the best in the NFL.

Let’s take the next step.

Adjusting for strength of schedule

In a nutshell, computer ranking systems take a statistic like margin of victory and adjust for strength of schedule. That’s it.

This adjustment is more critical in college football than the NFL. In college, teams divide themselves into conferences of vastly differing strength. SEC teams play a much more difficult schedule than their neighbors in the Sun Belt.

In the NFL, the salary cap levels the playing field, which makes adjustments for strength of schedule less important than in college football. However, you shouldn’t ignore these adjustments, especially for Carolina during the 2015 season.

All of my team rankings take margin of victory in games and adjust for strength of schedule. Here are the NFL rankings prior to the Super Bowl with Carolina’s opponents in italics.

1. Carolina, 10.2
2. Seattle, 8.3
3. Cincinnati, 7.0
4. Arizona, 6.9
5. Kansas City, 6.8
6. Pittsburgh, 6.1
7. New England, 6.0
8. Denver, 5.2
9. Green Bay, 4.6
10. Minnesota, 3.7
11. New York Jets, 1.0
12. Buffalo, -0.4
13. St. Louis, -0.5
14. Oakland, -1.0
15. Detroit, -1.0
16. Houston, -1.0
17. Baltimore, -1.7
18. Philadelphia, -1.9
19. Chicago, -2.0
20. New York Giants, -2.2
21. Washington, -2.3
22. Atlanta, -3.1
23. New Orleans, -3.1
24. San Diego, -3.6
25. Indianapolis, -3.7
26. Dallas, -5.6
27. Tampa Bay, -5.8
28. Jacksonville, -5.9
29. San Francisco, -6.2
30. Miami, -6.4
31. Cleveland, -6.8
32. Tennessee, -8.9

Carolina played three teams in the top half of my team rankings the entire season. Their 6 division games against Atlanta (22nd), New Orleans (23rd) and Tampa Bay (27th) didn’t present much competition. In addition, they faced the weak teams from the NFC East and AFC South in other games.

Despite this strength of schedule, Carolina still ranked first in these points based NFL rankings because of their large unadjusted margin of victory in games. To find a potential weakness for Carolina against Denver, we need to dig further.

Rankings pass offense and defense

The Power Rank algorithm can do more than rank teams on adjusted margin of victory. It can also rank offenses and defenses based on efficiency metrics.

To get a better insight into the match up between Carolina and Denver, let’s look rankings for pass offense and defense. To do this, we take yards per pass attempt and adjust for strength of schedule.

This list gives the pass defense rankings before Super Bowl 50, again with Carolina’s opponents in italics.

1. Denver, 5.1
2. Carolina, 5.5
3. Seattle, 5.5
4. Kansas City, 5.6
5. Cincinnati, 5.6
6. Houston, 5.8
7. Green Bay, 5.9
8. New England, 6.0
9. Oakland, 6.0
10. St. Louis, 6.1
11. New York Jets, 6.1
12. Minnesota, 6.1
13. Pittsburgh, 6.2
14. Philadelphia, 6.2
15. Arizona, 6.3
16. Baltimore, 6.3
17. Buffalo, 6.6
18. Chicago, 6.6
19. Tampa Bay, 6.6
20. Indianapolis, 6.7
21. Detroit, 6.7
22. Dallas, 6.7
23. Washington, 6.8
24. Atlanta, 6.8
25. Tennessee, 6.9
26. Jacksonville, 6.9
27. San Francisco, 7.0
28. San Diego, 7.0
29. Miami, 7.1
30. New York Giants, 7.2
31. Cleveland, 7.3
32. New Orleans, 7.9

Cam Newton only faced three solid pass defenses all season. Three!

I should note that Arizona’s pass defense would have made a fourth good pass defense before Carolina racked up 11.2 yards per attempt against them in the NFC championship game.

Carolina threw for almost 7 yards per attempt, 5th best in the NFL. However, strength of schedule adjustments drop Carolina to 11th in the pass offense rankings.

Also, Denver had the top ranked pass defense heading into the Super Bowl. The number next to each team gives a rating, or expected yards per pass attempt allowed against an average pass offense. Denver had a rating significantly better than second ranked Carolina.

Numbers over the eye test

It wasn’t easy trusting the numbers before Super Bowl 50. Everyone liked Carolina, as the markets closed with the Panthers as a 5 point favorite over the Broncos.

My member predictions, which use a number of metrics including the rankings discussed in this article, gave Carolina a 1 point edge. While this seemed a bit low, the match up of Denver’s pass defense against Cam Newton gave the Broncos hope.

In the game, Newton threw for 4.1 yards per pass attempt, well below his season average. This played a big role in Carolina’s loss to Denver, as numbers and analytics stood strong against the eye test in this game.