Core Principles of Sports Analytics

Do pitchers understand FIP (fielding independent pitching)?

One of the core principles of baseball analytics is fielding independent pitching.

This principle evolved from the research of Voros McCracken, who showed that pitchers do not control the hits they allow. Pitchers control strike outs, walks and to a lesser extent home runs. However, all major league pitchers allow a batting average on balls in play of .300 in the long run.

This idea evolved into FIP, a formula that turns a pitcher’s strike outs and walks into an ERA type statistic. The formula is a breakthrough in evaluating pitchers.

However, the formula isn’t the take home message about pitching analytics.

In the April 28th, 2014 issue of Sports Illustrated, Max Scherzer gave us the take home message.

…the advanced stats are great to look at for my long-term goals and what I’m trying to accomplish. It shows me there is an inherent failure in pitching. The luck involved, the factors you can’t control. You just have to let go of those and focus on the next batter, the next game. You can’t do anything about bloop hits. I didn’t understand that before, and now I do.

I nearly fell off my chair when I read that. He gets it. (So does Zack Greinke.)

Bill James could not have summarized FIP better himself.

How passing and rushing affect winning in the NFL

By Dr. Ed Feng 31 Comments

Before the Super Bowl, Bill Belichick told his Giants defense to let Thurman Thomas rush for 100 yards.

As David Halberstam writes in Education of a Coach, it was a tough sell before the 1991 Super Bowl against Buffalo. The New York Giants played a physical defense that prided itself on not allowing 100 yard rushers.

No matter, the short, stout coach looked straight into the eyes of Lawrence Taylor and Pepper Johnson and said, “You guys have to believe me. If Thomas runs for a hundred yards, we win this game.”

Just in case his players didn’t listen, Belichick took it upon himself to ensure Thomas got his yards. He took out a defensive lineman and linebacker and replaced these large bodies with two defensive backs. In football lingo, the Giants played a 2-3-6 defense designed to struggle against the run.

Did Bill Belichick go insane? I certainly thought so when I first read this story years ago.

However, analytics is on Belichick’s side. Let me explain.

Visual shows the importance of passing over rushing

When it comes to winning in the NFL, passing is king. Rushing hardly matters.

To quantify this, our football obsessed culture must look past misleading statistics such as rush yards per game. Teams with the lead run the ball to take time off the clock. Any team can rush for 100 yards if they run it 50 times.

To measure true skill, it is better to look at efficiency metrics like yards per attempt. A team can’t fake their way to 5 yards per carry by running the ball more.

Here, efficiency for passing and rushing is defined as yards gained per attempt on offense minus yards allowed per attempt on defense. Higher values indicate more team strength. Sacks count as pass attempts, and these negative yards lower pass efficiency on offense.

The visual shows the pass and rush efficiency during the regular season for all NFL playoff teams from 2003 through 2012.

From the left panel, playoff teams excel in passing, both throwing the ball on offense and preventing the pass on defense. Only 15 of 120 playoff teams in this era allowed more yards per pass attempt than they gained.

The visual also highlights teams that played in the Super Bowl. Eight of the ten Super Bowl champions were among the NFL’s elite in pass efficiency. However, excellence in the air does not guarantee playoff success. The New York Giants in 2007 and Baltimore in 2012 won the Super Bowl despite subpar pass efficiency.

Rushing hardly matters in the NFL

While the importance of passing in the NFL will not surprise anyone, the insignificance of rushing might. The visual for rush efficiency shows playoff teams as a random scatter of positive and negative values for their regular season statistics. A strong run game on offense and defense does not help a team make the playoffs.

Moreover, teams with a high rush efficiency do not suddenly become clutch in the playoffs. Almost half of the teams that played in the Super Bowl allowed more yards per carry than they gained. In 2006, Indianapolis won the Super Bowl while having the worst rush efficiency in the NFL. Green Bay in 2010 and the New York Giants in 2011 weren’t much better.

A guessing game of a team’s wins

Running the ball does not affect winning as much as you think. To illustrate this point, consider this guessing game. Suppose you want to guess how many games a team will win during the regular season. Without any other data, it makes sense to guess 8, the average number of wins in a 16 game season.

From 2003 through 2012, this estimate would be wrong by 3.1 wins. In technical jargon, 3.1 is the standard deviation of actual wins from the guess of 8. In normal people language, it says 2 of 3 teams will be within 3.1 wins of the guess. About two thirds of NFL teams won between 5 and 11 games between 2003 and 2012.

With the rush efficiency for each team, how much better does your guess get? The right panel of the visual below shows how rush efficiency relates to wins for every NFL team from 2003 through 2012. Simple linear regression gives the best fit line through the data.

The regression line gives a new guess about the number of games a team will win. For example, suppose a team has a rush efficiency of 0.6 yards per carry. Instead of guessing 8 wins for this team, the line gives 8.7 wins for this team.

How much better are these new guesses? Not much. The error only drops from 3.1 wins to 3.03 wins. In technical jargon, rush efficiency explains only 4.4% of the variance in wins. You might as well guess randomly.

The results get better using pass efficiency, as shown in the left panel. The error in estimating wins drops from 3.1 to 1.96. Pass efficiency explains 62% of the variance in wins in the NFL. The strong relationship is clear from the visual.

In college football, rush efficiency correlates more strongly with wins than in the NFL. Teams like Alabama, Stanford and Wisconsin have won with a power running game and a physical front seven on defense. The insignificance of running the ball is unique to the NFL.

Analytics gives a broad view of how passing and rushing affect winning. But to dig deeper, let’s look at specific teams and their strengths in these areas.

Indianapolis Colts

Under the leadership of GM Bill Polian and QB Peyton Manning, the Colts had a remarkable run from 2003 through 2010. They won at least 12 games each year before slacking off with 10 wins in 2010.

They achieved success through the air, ranking in the top 8 in pass efficiency each year. Peyton Manning and his offense played the bigger role, but the pass defense helped out some years. The Colts ranked in the top 10 in pass defense (yards allowed per attempt) from 2007 through 2009.

However, Indianapolis was really bad in the run game. Only once in this era (2007) did they gain more yards per carry than they allowed. As mentioned before, they were dead last in the NFL in rush efficiency in 2006 when they beat Chicago in the Super Bowl.

New England Patriots

New England won 125 games, 2 Super Bowls and played in 2 others during the 10 seasons covered by the visual. They followed the same script as Indianapolis: strong in passing, weak in rushing.

From 2003 through 2012, New England ranked in the top 10 in pass efficiency in each year except 2008 and 2012. In 2008, QB Tom Brady got hurt in the first game of the season. New England ended the season 13th in yards gained per pass attempt and did not make the playoffs, the only time this happened during these 10 years.

However, New England has never cracked the top 10 in rush efficiency. Coach Bill Belichick might not have seen the data presented here, but he gets the futility of rushing in the NFL. This understanding extends as far back as his days as defensive coordinator for the Giants.

Indianapolis and New England have built their teams around passing at the expense of rushing. They, along with New Orleans of recent seasons, have had success in winning games and Super Bowls. Now let’s look at teams that excel at rushing.

Minnesota Vikings

More than any other team, the Vikings dominate the ground game. They feature RB Adrian Peterson on offense and have tackles Pat and Kevin Williams clogging up the middle on defense. For the 6 years between 2007 and 2012, Minnesota has finished 1st in rush efficiency 4 of those years.

However, this strength has led to ups and downs in wins. Minnesota went 3-13 in 2011 despite leading the NFL in rush efficiency. The next season, they led the NFL again behind a monster season from Peterson, who made a remarkable return from knee surgery. The Vikings had 10-6 record that season.

The Viking’s best season over this stretch came in 2009. They finished 12th in rush efficiency that season. The difference? A QB named Brett Farve came out of retirement to play for Minnesota. The Vikings finished 7th in yards gained per pass attempt. They went 12-4 and came within a late turnover against New Orleans of playing in the Super Bowl.

San Francisco

The Niners started winning games when coach Jim Harbaugh became coach in 2011. However, they had their strengths before he arrived. Behind DE Justin Smith and LB Patrick Willis, San Francisco had an elite run defense. From 2007 through 2012, they never finished worse than 8th in yards allowed per carry.

This run defense didn’t help them win much the first 4 seasons, as the Niners won only 26 games. The pass defense never finished better than 15th during this time.

When Harbaugh arrived in 2011, San Francisco drafted LB Aldon Smith, a pass rush monster out of Missouri. They also signed CB Carlos Rogers, who had his first Pro Bowl season in 2011. The Niners have finished 9th and 3rd in pass defense in 2011 and 2012 respectively. This resulted in 24 wins during these two seasons.

How to evaluate NFL statistics

In Super Bowl XXV, Bill Belichick’s plan to let Thurman Thomas rush for 100 yards worked, maybe too well. Against a small defense designed to slow down the pass, Thomas ran for 135 yards on 15 carries, a staggering 9 yards per carry. In the second half, he broke off a 31 yard run for a touchdown.

The game ended when Bills kicker Scott Norwood sent a field goal attempt wide right. The Giants won the Super Bowl 20-19.

The Giants did not win the game solely because of Belichick’s defensive plan. The offense generated two long scoring drives in the second half that took time off the clock. And I would bet my life savings Belichick did not want his defense to allow that 31 yard touchdown run to Thomas.

But, as Halberstam discusses in Education of a Coach, Belichick did want the Bills to pick up small gains on the ground if it meant keeping Jim Kelly from throwing the ball. He understood that rushing means almost nothing to winning in the NFL.

If you’re going to remember anything from this article, it should be this: look at a team’s passing instead of rushing numbers to determine whether they will win games.

The One Thing Everyone Ought to Know about Sports Analytics

By Dr. Ed Feng Leave a Comment

In his first game as an NFL head coach, Josh McDaniels and his Denver Broncos faced a 7-6 deficit with 38 seconds remaining in the game. The Cincinnati Bengals had just scored a touchdown and had the Broncos pinned deep at their own 13 yard line. On 2nd down, Kyle Orton’s pass gets tipped right into the hands of Denver’s Brendan Stokley, who rumbled 87 yards for the winning score. The play has been immortalized as The Immaculate Deflection.

The 2009 Denver Broncos would win their first 6 games. This streak included games against New England, Dallas and San Diego. This impressive start prompted ESPN’s Tom Jackson to declare Coach McDaniels “one of the great ones”.

The 2009 Denver Broncos finished the season 2-8. They missed the playoffs when they lost to the Kansas City Chiefs, the 29th team in our rankings that year, in the season’s final game.

The 2010 Broncos started 3-9. Coach McDaniels was fired.

If you remember one thing about sports analytics, it should be this: never make a judgement after a small number of events.

How an average coach fares in an average league

Sports is inherently random. As The Immaculate Deflection shows, a team can win on a lucky bounce when they’ve only mustered two field goals the entire game. It’s not all that different from flipping a coin. To show the fallacy of looking at a small sample size, let’s look at how an average coach performs in an average league. Not too different from the NFL, this coach as a 50% chance of winning each game. Using a random number generator, these are the results of Coach Average’s first 50 games.

Just for the record, I was absolutely committed to generating this random sequence only once. There was no effort to find a sequence that had 6 wins in a row. Eight lines of Python code gave that result. Coach Average ripped off a sequence of 9 straight wins starting in game 19. Tom Jackson would be starting his petition for the Hall of Fame.

Moreover, Coach Average wins 31 of his first 50 games, 6 more than the expected 25 wins. In the modern era of the NFL, only Bill Belichick and Tony Dungy have better career winning percentages than the 62% of Coach Average. He looks extraordinary even over a sample of 50 games.

I actually generated a sequence of 200 coin flips, not knowing how many would fit in the image above. Coach Average won 106 of those 200 games for a 53% winning percentage. With a bigger number of coin flips, the winning percentage gets closer to the expected 50%. That’s the consequence of the Law of Large Numbers, the mathematical reason you should only draw conclusions after a large number of events.

What the famous “Hot Hand” paper says about all of this

In 1985, Amos Tversky, a Stanford psychology professor, and his colleagues published a paper called The Hot Hand in Basketball: On the Misperception of Random Sequences. Replace “The Hot Hand in Basketball” with “The Hot Coach in Football”, and this paper has everything to do with our discussion. There are two key results from their study.

First, they looked sequences of made and missed baskets for the two NBA teams and asked whether it looked any different from the random flipping of a coin. Does a made basket implied the next basket is more likely to go in? No. Were there more streaks of made baskets than one would expect from random? No. They even broke down the data into partitions of 4 consecutive shots to look at whether streakiness happened in short bursts. Did partitions with 3 or 4 made shots happen more than in the coin flipping model? Still no. It didn’t matter if it was field goals for the 1980-81 Philadelphia Sixers or free throws for the 1980-82 Boston Celtics. It just looked like a random sequence, much like our coin flipping model for Coach Average.

Second, they did a survey in which they gave people a sequence of X’s and O’s to represent made and missed baskets. This experiment isolates the random sequence from any sports related phenomena. The participant can no longer say they saw a streak or hits, or made baskets, because they are watching Andrew Toney or Kobe Bryant.

In a truly random sequence, a hit follows a hit with 50% likelihood. For these sequences, only 32% of participants called this random shooting while 62% called this streak shooting. People tend see streaks in randomness, just like you probably saw streakiness in the wins and losses of Coach Average. When the likelihood of getting a hit after a hit decreased below 50%, more people thought the sequence was random. People falsely expect to see alternating hits and misses in a random sequence.

Even without all the biases inherent in sports, people see order in randomness.

What this means for sports fans

Luck and random chance play a huge role in the short term. More importantly, luck tends to average out in the long term. So sports fans should not make rash judgements over a small number of events. Despite starting the season 2-10 or 4-10, Boston Red Sox fans should remember that they have a huge payroll and analytics on their side. When Albert Pujols hits 0.217 in this first 92 at bats, Los Angeles Angels fans should remember he’s one of the greatest hitters of his generation. Just wait a little bit.

Thanks for reading.