Podcast: John Urschel, NFL lineman, on football analytics and math research

On this episode of the Football Analytics Show, John Urschel, lineman for the Baltimore Ravens and Ph.D. candidate in mathematics at MIT, joins me for a wide ranging conversation. Topics include:

  • The NFL’s progress in using video tracking data
  • The football analytics problem John is working on
  • The acoustic operator in those Bose commercials with J.J. Watt
  • The breadth of math research in which John is engaged
  • The hidden factor in his success as an NFL lineman

John is an amazing human being, and I hope you enjoy this episode.

You also might be interested in an interview I did with John 3 years ago before he got drafted by the NFL.

To listen on iTunes, click here.

To listen here, click on the triangle.

Podcast: Rufus Peabody and the NFL Conference Championship Games

Rufus Peabody, ESPN’s predictive analytics expert who makes his living investing in the sports markets, joins me on the Football Analytics Show to discuss the NFL playoffs. He’s also half of the excellent Massey-Peabody rankings and predictions for football.

Among other things, we discuss the following.

  • The balance between a quantitative model and subjective adjustments
  • The one factor for Massey-Peabody that tips the balance against the spread in Green Bay at Atlanta
  • The new results on home field advantage that impacts the NFC title game
  • How our models differ on New England’s pass defense
  • Rufus’s book recommendation for those interested in randomness
  • The perils of small sample size in sports

You’ll find Rufus incredibly humble for someone who has had his success in sports analytics, and I’m lucky to consider him a friend.

To listen on iTunes, click here.

To listen on the site, click on the play button.

How Andy Reid wins football games with interceptions

Andy Reid’s teams throw a low rate of interceptions. The visual shows how his teams in Philadelphia and Kansas City have had a lower than NFL average interception rate (interceptions divided by attempts) in all but 3 of 18 seasons.

Reid’s Eagles had a particularly good stretch of pick suppression from 2000 through 2004. Led by quarterback Donovan McNabb, Philadelphia never won fewer than 11 games in any of those 5 seasons.

Despite a decreasing interception rate across the NFL, Reid has continued to beat the NFL average over the past four years in Kansas City. Led by quarterback Alex Smith, the Chiefs have won 43 regular season games and never dipped below 9 wins in any one season.

The randomness of turnovers

The visual goes against the typical quant narrative that turnovers are random.

For example, I’ve shown this visual that shows the relation between interception rate the first 6 games of the college football season versus the remainder of the season.

The lack of correlation between these quantities shows that you can’t predict a team’s interception rate later in the season based on the same quantity during first 6 games.

This suggests interceptions are random, and a team has a 50% chance to have a better or worse than average interception rate. However, if you assume this for Andy Reid’s teams, there’s only a 0.37% chance his teams would have had 3 or fewer seasons with a below average interception rate.

Randomness certainly plays a role in interceptions. No one who has ever seen a tipped pass fall into the hands of a defender should doubt that.

However, the Reid visual suggests that some coaches can suppress interceptions over a very large sample of games.

Steelers at Chiefs

This has implications in predicting the outcome of the Steelers at the Chiefs playoff game this weekend.

Kansas City doesn’t seem like much of a Super Bowl threat with the 16th and 11th ranked pass offense and defense, respectively, by my adjusted yards per attempt. I use these pass efficiency numbers to evaluate teams for two reasons:

  • My research shows the importance of passing over rushing in the NFL.
  • Turnovers have little impact on yards per pass attempt.

However, if Kansas City is truly skilled at not throwing interceptions, then these pass efficiency numbers will underestimate their team strength.

Team rankings based on adjusted margin of victory might be a better way to evaluate Kansas City. Their low interception rate will impact margin of victory, as I’ve found that an interception is worth 5 points in the NFL.

My member numbers combine both pass efficiency and margin of victory to make Kansas City a 1.3 point favorite against Pittsburgh. However, my team rankings based on only points would make Kansas City a 3 point favorite.

I interviewed Ben Alamar on the Football Analytics Show this week, and his FPI (Football Power Index) makes the Chiefs nearly a 5 point favorite. They use an expected points added, a metric which accounts for interceptions but their own twist. To listen to that part of the discussion, go to 14:50 of my interview with Ben Alamar.

How often Jim Harbaugh’s 4th down decisions agree with analytics

This is a guest post from Tony Kaminski, a recent University of Michigan graduate and engineer. You can follow him on Twitter or check out his site Big House Analytics.

How does Jim Harbaugh’s fourth down decisions stack up to analytics?

The numbers say that coaches should go for it on fourth down way more than they do. Berkeley professor David Romer started this research back in 2006, and Brian Burke, now with ESPN, has performed the most complete study based on NFL data.

My interest in this topic as a Michigan fan began with a conversation with Craig Ross, a member of WTKA MGoBlog Roundtable and author of two books on Michigan sports. In his book The Search for the Unified Field Theory (Football Version), he writes

So here’s the deal. Einstein’s follies aside, I believe there is a unified field theory in football, a fundamental set of ideas or equations that can help to explain “what happened” in a football game, why the game was won or lost. More than this, I think this theory allows a coach to make choices and structure his team and program in a way that squeezes out a few more wins than might otherwise be the case.

And therein lies our aim: finding this unified theory. Craig and others have good reason to believe that not punting very much, or at least substantially less than coaches tend to do currently, is a part of applying this theory to winning more games.

The natural starting point of this conversation is the work of David Romer, the Berkeley economist. An excerpt from a New York Times profile of Romer explains the motivation looking at 4th downs:

Romer, a lifelong sports fan who is a professor at the University of California, Berkeley, came up with the idea to rigorously examine fourth-down plays after listening to a radio broadcast of an Oakland Raiders game in his car about a decade ago. Although the Raiders had the ball in striking distance of the end zone, one of the commentators remarked that they would be smarter to kick a near-certain field goal rather risk going for a touchdown.

“I am pretty analytic,” Romer recalled telling himself. “That is a pretty shallow way of thinking about it.”

So, after stewing over the idea for a couple of years, he set out to tackle the great fourth-down debate.

Romer’s work has been largely met with skepticism in coaching circles because the difficulty in overcoming conventional wisdom. Coaches at big-time universities or professional franchises with nine-figure payrolls often stick to the proverbial book — and why not?

If you fail underneath the brightest lights, it’s easier to defend yourself when your decisions echo the conventional wisdom. Wins come at a premium, and no career has less job security or more turnover than football coach. So you do everything you can to hold onto that gig, because they don’t come around often.

But doing great things, in the private sector or on the football field, involves bucking convention. Romer has shown this means punting less. Way less.

You can read this article for a quick tutorial on Romer’s work and its application. While the New York Times model doesn’t explicitly reference the professor, their “4th Down Bot” comes to a very similar conclusion, just with more data to back it up. It boils down to this:

Especially near the middle of the field, coaches need to be more aggressive. In this article, we’ll look at Jim Harbaugh’s fourth down decisions since his arrival at Michigan. A few points on this analysis.

  • While Romer and the New York Times used NFL data, Romer believes the findings also apply to the college game. (Craig recounts an email conversation that him and the professor had in his book where Romer explains why.)
  • Bill Connelly’s S&P has designated garbage time as being up by 25 points or more in the second quarter, 22 or more in the third, or 17 or more in the fourth. For the 2015 analysis, I filtered this out arbitrarily, in a more liberal manner than Bill does, based on my recollections of the individual games. In 2016, I followed Connelly’s rules.
  • This is not an indictment of Harbaugh’s decision making on fourth down, merely an analysis of how closely his decisions echo Romer’s advice. There are specific team, opponent, or situation-variant times where it would be best to not follow the chart above.
  • This data does not reflect the final outcome of the decision made but only the decision to go for a fourth-down conversion or punt itself.

Let’s first look at Harbaugh’s 2015 fourth down decisions in these tables. The color green designates a decision that follows what analytics recommends. Yellow means it did not but was defensible given the score, situation, or general trend of the game (more on that later). Red designates a time when Harbaugh probably should have gone for it on fourth down opposed to punting.

Let’s discuss a few of the decisions marked yellow, or ones I found defensible given the game situation. (Maybe I was a bit too friendly to our coach, I’m a fan and proud alum, after all — leave a comment if you disagree with any of my designations, and we’ll continue the conversation).

For instance, Michigan punts on fourth and seven from the Michigan State 38 with 1:12 to go in the first half of a 10-7 game. That’s not technically the correct decision but consider the following:

  • Michigan State is out of timeouts
  • Michigan had allowed exactly seven points over the prior 17 quarters
  • Michigan had one of the best punters in college football in 2015

Michigan State runs three quick plays without scoring before halftime.

Against Northwestern, it’s fourth and nine from their 34 with 4:19 to go in the second quarter. Michigan has a 21-0 lead. The chart says to try a field goal but that’s too deep for then-freshman kicker Kenny Allen, who had missed a 44-yard attempt earlier in the season. Michigan punts, and then Jourdan Lewis houses an interception on the ensuing possession that erases any doubt about the outcome.

This isn’t to say I agree with all of the times Harbaugh deviated from the research. I was upset by the decision to punt from Ohio State’s 36 on fourth and five less than five minutes into that game. Given the talent discrepancy between the Wolverines and Buckeyes, who littered the first round of the NFL draft (literally, probably), points would be at a premium, and we passed up an opportunity to get some early.

Ultimately, Harbaugh had a 77.8% “success rate” — out of Michigan’s 54 fourth down decisions, Harbaugh went with analytics or I found the punt defensible on 42 of them. If you want to go by the book (meaning the punts I deemed as defensible but not Romer-approved counting against our success rate instead of for it), our score is 59.3% (32 of 54).

The following table shows my results for the 2016 season through the Ohio State game.

On the following fourth downs, Harbaugh went for the first down.

This analysis shows a 91.3% success rate (42 of 46). The results were so staggering that I had to have a friend double check my work. If you went purely by the numbers, our success rate is 76.0% (35 of 46) . By either metric, this is a substantial improvement from last season.

A couple notes:

  • Michigan blew out seven of their 12 regular-season opponents, so this chart was significantly easier to compile than last year’s.
  • I did not chart the “defeat with dignity” portion of the Michigan State game, the final offensive play against Indiana (it was a full-blown blizzard in Ann Arbor at that point and IU had effectively conceded), or the fourth-down attempt in overtime against Ohio State since we obviously didn’t have a choice.
  • There were only four punts I gave the “red” designation, and even then, I felt that I may have been being unreasonably harsh on Harbaugh, since these were coming up correct at a much higher clip than last season.
  • Michigan was 12 for 18 on fourth-down conversion attempts in 2016 compared to six for 16 in 2015. More of these came in garbage time this year than last, hence the disparity in charted attempts.

Harbaugh’s decision making already agreed with analytics a good bit in 2015, but 2016 represents an even bigger step forward. I have a few hypotheses on why.

First, Michigan has a larger analytics team than under previous regimes. It’s not hard to believe that they had a part in this trend.

Second, Harbaugh had a different situation on defense these two years. In 2015, the defensive production fell off with the injury of Ryan Glasgow, most notably when Indiana gashed Michigan for 307 rushing yards in Bloomington and the dreadful Ohio State game.

This was not the case in 2016. Don Brown’s defense was stout all season long, which gave Harbaugh the confidence the defense could get a stop on a short field.

Does Aaron Rodgers draw more pass interference penalties?

On my recent appearance on Beating the Book, we were discussing Aaron Rodgers and why he wasn’t playing as well. Host Gill Alexander thew out the idea that Rodgers’ performance might not seem as bad if we included drawn pass interference penalties.

I dug into the 2016 play by play data through week 10 to find out. Rodgers has drawn 7 pass interference penalties, just above the team average of 6.

Drawing pass interference penalties doesn’t seem like a skill, as Drew Brees has 3 while Blake Bortles and Ryan Fitzpatrick have 12 and 11 respectively.

Rodgers does seem to draw pass interference penalties deep down the field. Here are the yardage gains on these penalties: 44, 18, 30, 40, 13, 28, 66. If you include these plays, it would help a pass offense that has averaged 5.8 yards per attempt, 26th in the NFL.

Here are the full results for defensive pass interference penalties for all teams during the first 10 weeks of the 2016 season.

  • Arizona, 7.
  • Atlanta, 6.
  • Baltimore, 6.
  • Buffalo, 4.
  • Carolina, 6.
  • Chicago, 4.
  • Cincinnati, 9.
  • Cleveland, 6.
  • Dallas, 2.
  • Denver, 9.
  • Detroit, 9.
  • Green Bay, 7.
  • Houston, 6.
  • Indianapolis, 5.
  • Jacksonville, 12.
  • Kansas City, 1.
  • Los Angeles, 5.
  • Miami, 2.
  • Minnesota, 4.
  • New England, 5.
  • New Orleans, 3.
  • New York Giants, 7.
  • New York Jets, 11.
  • Oakland, 12.
  • Philadelphia, 3.
  • Pittsburgh, 5.
  • San Diego, 8.
  • San Francisco, 3.
  • Seattle, 6.
  • Tampa Bay, 7.
  • Tennessee, 9.
  • Washington, 6.