THE POWER RANK

Who else wants to predict the 2013 college football season with confidence?

By 12 Comments

How good is your team this year?

Is your team the best in its division or conference?

Will your team win the rivalry game this year?

There is no shortage of media outlets that will attempt to answer these questions before a game has been played. At my local Barnes and Nobles this August, there were 6 different preseason college football magazines, including Athlon, Lindy’s and Phil Steele.

The preseason Coaches and AP poll will also give you their opinion on your team.

These established brands have been around since the dinosaurs. They have years of experience in making preseason rankings.

So why would you look at another preseason rankings, especially from an upstart like me? Who cares about a regression model using past team performance, returning starters and turnovers when you could just lean on these established brands?

tpr_preseason_accuracy

To put this in perspective, consider these results from The Prediction Tracker, a site that tracks the accuracy of predictions systems.

Since 2005, the famous Sagarin Predictor rankings have picked the winner in 61.9% of bowl games. The opening line from Vegas odds makers predicts 59.9%. Both Sagarin and the odds makers can consider games during the season, information not used in The Power Rank preseason rankings.

For the entire season, Sagarin Predictor has picked the winner in 73.6% of games. The Prediction Tracker has data on 35 systems for each season since 2005. The season accuracies range from 74.5% to 68.1%. Again, these systems can use games from the season, information not used in my preseason rankings.

To check out The Power Rank’s preseason rankings for 2013, click here.

Let’s look at two factors in calculating this accuracy.

Accuracy in past seasons

While I developed this regression model for preseason rankings during the summer of 2013, I went back and calculated the results during previous seasons.

This requires some care in only using data from previous seasons in the calculation. For example, the 2005 preseason rankings only uses data from the 2004 season and before.

A rating for each team

In calculating the predictive accuracy, I assume no home field advantage in a bowl game. Hence, the higher ranked team is predicted to win, which was accurate in 60.6% of bowl games.

However, home field advantage complicates a prediction for regular season games. Fortunately, The Power Rank’s preseason predictions assign each team a rating, which gives an expected margin of victory against an average FBS team.

To make a prediction on a regular season home game, three points are added to the home team before picking the team with the higher rating to win. This system predicted 70.5% of college football game winners over the entire season.

Uncertainty in any preseason rankings

These preseason rankings still contain a great amount of uncertainty. They do not consider a single game during the regular season. Even though these preseason rankings predicting over 70% of game winners the last 8 years, a team’s rank and rating can change drastically by the end of the season.

Let’s quantify this. These preseason rankings are based on a regression model that considers team history over the previous 4 years, returning starters and turnover margin. These predictors are used to fit the final team ratings for the next year. Hence, it makes sense to ask how much the predictions deviate from the actual ratings calculated from games that season.

In the eight seasons since 2005, 72.2% of teams have finished within 7 points of their predicted rating.

Seven points in a team’s rating can make or break its season. For example, Nebraska, 25th in the preseason rankings, moves up to 6th ahead of Georgia with an additional 7 points. However, they drop to 58th behind UCF if my estimate is 7 points too high.

Let’s look at the 3 factors that go into the regression model for these preseason rankings.

Team History

College football programs are large, lumbering institutions that do not change overnight. Alabama will always have a rich tradition that started with Bear Bryant and continues through Nick Saban. Nothing Rice can do will give my Owls the tradition of the Crimson Tide. Fans will trickle into Rice Stadium while Alabama will sell out the 100,000 plus seats in Bryant Denny.

Because of this persistence, the regression model uses a team’s rating in The Power Rank from the last 4 seasons to predict next season. From studying previous seasons, these ratings have an overwhelming effect on how a team will perform next season. Those 3 national championships that Alabama has won over the past 4 seasons matter, and this history gives the Crimson Tide their #1 ranking.

Returning Starters

A team greatly benefits from having players with past experience on the field. For example, Stanford only finished 9th in last year’s rankings despite their Rose Bowl win. However, the Cardinal have 15 starters returning, including many from a stellar defense. With the continual development of QB Kevin Hogan, who started the last 6 games of the season, Stanford should improve from last season’s results.

Turnovers

Big plays like fumble recoveries have an enormous impact on games. However, predicting future turnovers is as difficult as telling whether a coin will land heads or tails. It’s a counter intuitive result, as most fans believe in the big play defense that forces turnovers. But turnovers forced regress strongly to the mean.

The Oklahoma State defense discovered this regression last season. In 2011, the Cowboys forced 44 turnovers on their way to a 12-1 season and Fiesta Bowl win. Articles were written about how they practice forcing turnovers. In 2012, the Cowboys forced 22 turnovers, very close to the FBS average. They finished 8-5 last season.

While a positive turnover margin (take aways minus give aways) will enhance a team’s rating one year, this luck will most likely disappear the next season. This doesn’t always happen, as Rich Rodriguez found out during his 3 year tenure at Michigan. His teams never did better than a -10 turnover margin. However, the math is strong enough to include this factor in the regression model.

To check out the preseason rankings for 2013, click here.

Let’s look at two teams that jump out of these rankings.

Ohio State

With their unblemished 12-0 record from last season, most fans and pundits view the Buckeyes as the team to end the SEC’s run of seven national titles. However, my numbers think different, placing Ohio State 13th. They ended last season 14th, as the Buckeyes had a difficult time beating poor teams like UAB and Indiana. Top ranked teams like Alabama punished these types of opponents.

To enhance this comparison of Ohio State and Alabama, consider margin of victory. Last season, Ohio State had an average margin of victory of 14 points. This was about half of the 28 points that Alabama posted despite their loss to Texas A&M. In addition, Ohio State played a weaker Big Ten schedule than Alabama’s in the SEC.

I think 13th is a little low for Ohio State, particularly with the talent that coach Urban Meyer has recruited over the last two seasons. However, it’s unlikely Ohio State makes the jump to national title contender.

USC

A year ago, a USC was the media darling of the college football. The Trojans finished the previous season strong, including a dramatic win over Oregon that ended the Ducks’ national title hopes. Then QB Matt Barkley turned down the riches of the NFL to return for his senior season, which propelled the Trojans to #1 in the preseason AP poll.`

USC ended the 2012 season 7-6, far from the title contention that most expected for the Trojans. They lost a number of close games to good teams like Stanford, Oregon and Notre Dame. Then the wheels fell off when Barkley got hurt and Georgia Tech dismantled them in the Sun Bowl. This sent expectations for 2013 into the toilet, as some rankings do not even have USC as a top 25 team.

Of course, this is just as ridiculous as their #1 rank from last season. USC has placed in the top 20 of my rankings the last 3 seasons. Even with their 7-6 record last year, the Trojans ended the year 20th. This team history matters, and my preseason rankings have USC at 10th.

Accuracy in predicting conference winners

I don’t know anyone else that has checked the accuracy of preseason rankings in predicting game winners in college football. If you know of any studies, please let me know in the comments.

However, Stassen has been tracking the accuracy of how preseason magazines predict the final standings in division and conferences. Their site is definitely worth checking out.

My rankings do generate the winners of conferences through a Monte Carlo simulation. I wrote about this previously when complaining about the injustice of schedule in college football. However, I have not yet checked the accuracy of these results against the data of Stassen. It will probably have to wait until next season.

Thanks for reading. And if you still haven’t checked out the preseason rankings for 2013, here is that link one last time. 🙂

The Top 25 College Football Teams of 2013 by Recruiting Rankings

By Leave a Comment

Nick_Saban_StatueRecruiting rankings do matter.

Each year, Rivals assigns a rating or points value to each school that describes the talent of the players who signed a letter of intent. For Sports Illustrated, we developed a model that takes the Rivals ratings and predicts future team performance. To compare the rankings from this model with the preseason AP poll, we looked at which rankings better predicted the final AP poll.

The Rivals model did as good or better than the preseason AP poll on 46 out of 100 teams over the last 4 years. This is remarkable given the limited information the recruiting model has compared with the writers that vote in the AP poll.

To get the full story on SI.com, click here.

Before we count down the top 25 teams for 2013, we note the following about this regression model.

  • The regression model has learned from the past by relating recruiting ratings to the team ratings from The Power Rank algorithm. For example, recruiting data from 2009 to 2012 were fit to The Power Rank’s results for the 2012 season. Our team ratings from the regular season have picked the winner in 62.8% of bowl games over the last 11 years, a better percentage than the Vegas line (62.2%). Part of the accuracy of the recruiting model depends on our team ratings.
  • We used the final AP poll as the measuring stick for the accuracy of the Rivals model and the preseason AP poll. This poll has problems, as it strongly considers wins but almost ignores margin of victory. However, it was the most relevant measurement of team strength for a general college football audience.
  • The 100 teams we looked at were the top 25 teams in the preseason AP poll over the last 4 years. This puts the recruiting model in a better light, since this set of teams didn’t include teams highly ranked by the recruiting model but outside the AP top 25. For example, the recruiting model had Auburn 7th heading into the 2012 season. The preseason AP poll had Auburn 28th, which more accurately predicted their disasterous season.

With the passing of National Signing Day 2013, we now have the Rivals ratings to predict the 2013 college football season. The predictions are based the past 8 years of team performance.

25. Stanford. The Cardinal only had 12 scholarships to offer incoming freshmen. Since the number of recruits directly affects the Rivals team rating, Stanford only had the 61st ranked class this year, a far cry from their 5th ranked class a season ago.

24. Miami (FL). Miami coach Al Golden can’t walk off campus without tripping over a highly touted high school player. However, impending NCAA sanctions made recruiting difficult this year, as they had the 44th best class, much worse than their 9th ranking a year ago.

23. Virginia Tech. Despite two down years, coach Frank Beamer still recruited the 22nd ranked class. The Hokies need QB Logan Thomas’s passing to improve or groom a better passer.

22. California. The Bears continue to recruit well despite the coaching change from Jeff Tedford to Sonny Dykes. However, this ranking is probably too high. The Cal offense will be learning a new spread system under Dykes, while the defense lost coordinator Clancy Pendergast to USC.

21. Washington. The Huskies have continually improved their recruiting rank over the last 4 year, rising from 28th in 2010 to 18th in 2013. Hopefully, some of the linemen recruiting during this time will give QB Keith Price better protection next season.

20. Nebraska. The Cornhuskers pulled in the 17th best class, by far the 3rd best class in the Big Ten. Unfortunately, Michigan and Ohio State were way ahead, and Nebraska has to travel to Ann Arbor this season.

19. South Carolina. With all the media chatter about the talent in Columbus and the draw dropping hits from Jadeveon Clowney, you might think South Carolina is a top 10 recruiting team. In reality, their 16th ranking in 2013 was their best over the last 4 seasons.

18. Tennessee. The Rivals model had the Vols 13th in the nation last season, which was way too high. With new coach Butch Jones taking over, this year’s 18th ranking is also probably too high.

17. Oregon. Over the last 4 years, the Rivals model has consistently underrated the Ducks. The preseason AP poll was more accurate each year. Will this continue after head coach Chip Kelly handed the program over to Mark Helfrich?

16. UCLA. In talent rich Southern California, the Bruins always recruit well. They finally lived up to that talent last season under first year head coach Jim Mora. Their 11th ranked class this year should continue this trajectory.

15. Texas A&M. New coach Kevin Sumlin is getting tons of credit for recruiting the 10th best class this year. Of course, it helps that Johnny Manziel (a 3 star recruit) led the Aggies to their best season in more than a decade.

14. Mississippi. The Rebels and coach Hugh Freeze had a magical signing day, landing two top 10 recruits on the offensive and defensive line. Their 7th ranking is by far their best since 2002. However, three of their rivals in the SEC west are ranked higher in these rankings.

13. Clemson. With their dramatic, come from behind win against LSU in the Chick-fil-A Bowl, the Tigers will surely be an overrated top 10 team in the preseason AP poll. This ranking in the teens seems more appropriate.

12. Oklahoma. In the past 12 years, the Sooners have recruited a top 10 class 7 times. However, none of these top 10 classes have occurred during the last 3 seasons. Coach Bob Stoops needs more talent on the defensive line to start contending for national titles again.

11. Texas. Even Mack Brown can’t recruit after 3 subpar seasons. The Longhorns had the 23rd ranked class, their worst since 2002. Their usually excellent defense really needs help after last season.

10. Georgia. How would the Bulldogs recruiting class would have fared if the coaches had told QB Aaron Murray to spike the ball during the waning moments of the SEC championship game? A win over Alabama would have landed Georgia in the national title game against Notre Dame. Instead, they had the 12th best class, a fine rank but the second worst for coach Mark Richt over the last 12 years.

9. Michigan. After a disasterous 3 seasons under Rich Rodriguez, Brady Hoke has turned around this program both on the field and recruiting trail. The Wolverines notched the 5th ranked class after finishing 7th last year. These are their two best ranked classes over the last 8 years.

8. USC. While we kept hearing about the defections from the Trojan’s class, no one mentioned that coach Lane Kiffin still had five 5 star recruits coming to campus, more than any other school (yes, even Alabama). Lack of talent will not be a problem for USC.

7. Auburn. Yes, feel free to call BS on this one. The Tigers continue to recruit well. But unless the next Cam Newton and Nick Fairley show up on campus this fall, Auburn will not return to elite status this season under new coach Gus Malzahn.

6. Florida State. Call BS on this one at your own risk. Despite disappointing loses to NC State and Florida this year, expect the talent rich Seminoles to exceed expectations next season.

5. LSU. It’s a bit shocking that top recruit Robert Nkemdiche picked Ole Miss over LSU. There’s more certainly in winning games in Baton Rouge. Still, coach Les Miles landed the 6th best class in the nation.

4. Notre Dame. The Fighting Irish took full advantage of their undefeated regular season and landed the 3rd best class in the nation. While we should expect Notre Dame to be good next year, 4th is probably too high.

3. Ohio State. The Buckeyes will not surprise anyone this year. In coach Urban Meyer’s first season, Ohio State started the year 18th in the preseason AP poll. Now, with an undefeated season and a 2nd ranked recruiting class, anything but a national title will be a disappointment.

2. Florida. Last year, first year coach Will Muschamp landed the 3rd ranked class despite finishing 7-6. Somehow, a 11-2 season this year got the Gators the 4th ranked class (although they did top the rankings before National Signing Day). Talent is never a problem at Florida.

1. Alabama. Duh. What did you expect? After going 7-6 in his first year at Alabama, coach Nick Saban still recruited the top ranked class in 2008. This started a streak of top ranked classes for Alabama in 5 of the last 6 years. The rest of the SEC should resort to a voodoo consultant to bring bad turnover luck to Alabama next season.

Outlook

No one should take these rankings too seriously. With the short season and the youth of the players involved, college football is incredibly difficult to predict during the preseason. And these rankings have their problems. There’s an incredibly high likelihood that Boise State will be better than the 60th best team in the nation next year.

However, these rankings are still useful, and not only because they are six months ahead of the preseason AP poll. As with all predictive analytics, use these rankings has a guide to help navigate expectations for next season.

Moreover, this is only the beginning of our preseason college football predictions. To keep up to date, sign up for our free email newsletter below.

Thanks for reading.

How To Use Simple Division to Evaluate Notre Dame at USC

By Leave a Comment

The BCS calculus for Notre Dame is simple. Beat USC on Saturday and play in the national championship game. Can the Fighting Irish do it?

Most of the headlines focus on Notre Dame’s defense. Led by linebacker Manti T’eo, the Fighting Irish have allowed 10.1 points per game, best in the nation. With the injury to USC quarterback Matt Barkley, Notre Dame’s defense will most likely have a good day.

But what about Notre Dame’s offense?

Notre Dame’s Offense

The raw statistics suggest Notre Dame has a poor offense. For example, the Fighting Irish pass for 211 yards per game, a number that includes negative yards from sacks. Their 74th ranking places them below average nationally.

However, total yards per game is a misleading statistic.

Yards per game depends on how many times a team throws the ball. Notre Dame doesn’t throw the ball as often as the spread offense of West Virginia. Using simple division, we divide total pass yards per game by the number of attempts to obtain a better metric of pass offense. Notre Dame averages 6.83 yards per pass attempt, 51st best in the nation.

Simple division turns a below average pass offense into an above average one.

Dividing total yards by the number of plays lets us account for the pace of the football game. It is analogous to the tempo free statistics that Dean Oliver introduced into basketball. Looking at points per possession shed a new light on offenses that didn’t fast break and looked for the best possible shot late in the shot clock. Yards per attempt is the first step in incorporating these ideas into football.

Match Up With USC’s Defense

At The Power Rank, we take yards per pass attempt and adjust it for strength of schedule. Since Notre Dame has played top pass defenses such as Stanford, Michigan State and Oklahoma, our algorithm bumps up their pass offense from 51st to 28th. Their rating of 7.04 gives a predicted yards per attempt against an average pass defense.

Below, we show how our adjusted numbers for Notre Dame’s offense match up with USC’s defense. A better defense has a blue dot further to the right. Then the unit with the dot further to the right is predicted to have the advantage in the match up. By overall yards per play, Notre Dame’s offense enjoys a slight advantage over USC’s disappointing defense.

The Power Rank shows how teams match up with data visualization.

These visuals appear on our interactive team pages, the heart of our premium college football product. For more information, click here.

Outlook

Our team rankings predict a very even game (USC by 0.3 points). However, these numbers reflect a USC team with quarterback Matt Barkley. The Vegas line has Notre Dame as a 6 point favorite.

Is the injury to Barkley worth 6 points? We’ll find out Saturday.

Thanks for reading.

Forecasting the College Football National Championship Game

By 5 Comments

And then there were six
Who will play in the big game?
Behold their chances

The Power Rank, College Football, Week 9, 2011

On the last Saturday in October, college football offered some huge conference match ups. In the Big Ten, Wisconsin traveled to Ohio State to face the Buckeyes at The Horseshoe. Down south, the SEC pitted Georgia against Florida amidst The World’s Largest Outdoor Cocktail Party. But the most important game on Saturday, as validated by the presence of ESPN’s College Gameday, was Stanford at USC. It wouldn’t be surprising if USC were attracting all the media attention given their success this decade. But times have changed. Gameday showed up in Los Angeles because of an undefeated Stanford team. The Cardinal are one of a few teams in contention to play the SEC champion in the national championship game.

Before the game, USC coach Lane Kiffin won the media BS award of the week by calling Stanford quarterback Andrew Luck perfect and asking his interviewer for any ideas about how to stop Stanford’s offense. Kiffin didn’t need to consult his dad, USC’s defensive coordinator, to know that slowing down Stanford’s running game and getting pressure on Luck would mitigate Stanford’s offense. Or maybe he did. Either way, USC held Stanford to 3.9 yards per carry (186 total), much lower than their 5.6 yard average for the season. USC also sacked Luck twice. While this sack total might not sound impressive, Stanford’s offensive line, which features 2 potential first round picks, has allowed only 2 sacks all season. Despite this defensive performance, Stanford escaped with a 56-48 win when Ben Gardner’s right arm swatted the ball away from USC’s Curtis McNeal to end the third overtime.

Stanford remains undefeated with the win, but Kansas State and Clemson dropped out of the national championship picture with losses to Oklahoma and Georgia Tech respectively. With only 6 undefeated teams remaining, The Power Rank can update the projections that each of these teams remain undefeated. However, this week, we make the transition to rankings that only use games from this year. Up to this point, the rankings included all games in a one year window, with this year’s games counted twice. This drastically overrated teams like Auburn, who had a huge 56-17 win over South Carolina (19) and a 3 point win over Oregon (5) at the end of last year. With the loss of key players, coach Gene Chizik has found this season much more difficult. This change drops Auburn’s rating by 8 points, moving them from 12th to 38th. Perhaps this lower ranking isn’t a surprise given Chizik’s 5-19 record at Iowa State before becoming Auburn’s coach. Moreover, this change also improves Oregon’s rating by 2 points. These changes affect our forecast since Alabama plays Auburn while Oregon travels to Stanford. Here are the chances that each of 6 teams remains undefeated.

Boise State: 66.6%
Stanford: 41.7%
Alabama: 39.8%
Oklahoma State: 39.7%
LSU: 34.3%
Houston: 32.2%

With these changes, Alabama’s likelihood of going undefeated gets closer to Stanford’s than in our previous analysis. Alabama has the toughest remaining game, a 58.7% chance of beating LSU, but Stanford still faces Oregon with a 67.6% chance of winning. The changes in the rankings also recognize Houston as a completely different team with the return of quarterback Case Keenum this year. The Cougars move from 45th to 23rd and have a 32.2% chance to remain undefeated, much higher than the 8.7% we predicted earlier. This analysis includes a Conference USA championship game against Southern Miss.

The Power Rank can also estimate the likelihood that LSU and Alabama have a rematch in the national championship game. For this to happen, the remaining 4 unbeaten teams must each lose at least one gamen and the winner of the Tuscaloosa Tussle must go undefeated while the loser ends the season with only that one loss. The new rankings give this a 15.3% chance, quite a bit higher than the 4.9% chance we forecasted last week. Of course, these conditions do not ensure a rematch, since Oklahoma reenters the picture if they beat Oklahoma State and finish the season with one loss.

A more interesting scenario arises if we tweak the last situation so that Boise State goes undefeated. The winner of Alabama LSU and Boise State would be the only unbeaten teams, while the loser of the Tuscaloosa Tussle finishes the season with one loss. This is my personal favorite scenario since the SEC would secede from college football if an undefeated Boise State made the national championship game over a one loss Alabama or LSU team. The Power Rank gives this a 10.1% likelihood, about a one in ten chance for chaos.

Our analysis also produces some surprising numbers. As dominant as Alabama and LSU have been all season, there is a 30.4% chance that they both lose at least one game. While this number might seem absurdly high, consider that one of these two teams will lose on Saturday when they play each other. So 30.4% is the probability that the winner loses one of three remaining regular season games or the SEC championship game. The Power Rank suggests taking up a friend who will pay 10 to 1 if LSU and Alabama will both lose at least one game. Of course, no undefeated SEC teams greatly helps Boise State make the national championship game. The chance that Boise State remains the only undefeated team at the end of the regular season is 5.5%. They might make the championship game in other scenarios, but the system won’t lock them out as the only undefeated team.

Have a scenario of which you’d like to have the probability? Leave us a comment, please. Thanks for reading.

Stanford (2) will beat USC (27) by 18.9 on the road

By Leave a Comment

The final The Power Rank for Stanford and USC over the last 12 years.

A historical look at Stanford and USC.

In 2000, USC hired Pete Carroll as head coach. While he was their fourth choice to take the job, Carroll created a college football dynasty at USC through competitive recruiting and a jovial atmosphere. The Trojan’s execution in big games was breath taking at times. However, USC was in decline by his last year in 2009, and he left the school with NCAA sanctions when he left for the NFL. In his successor Lane Kiffin’s second year, USC’s rating has slipped from last year despite a 6-1 record.

In 2006, Stanford hit rock bottom by going 1-11. The new Stanford Stadium didn’t seem to inspire the players or soon to be fired head coach Walt Harris. Jim Harbaugh took over in 2007 and created a physical mentality that has turned Stanford into one of college football’s elite teams. Stanford’s -7.1 rating after the 2006 season became 23.8 by the time Harbaugh left for the NFL after the 2010 season, a staggering 30.9 point increase in team strength. David Shaw took over as head coach this year and increased Stanford’s rating to 29.3 in his first 7 games, although they haven’t faced an opponent ranked higher than 45th in The Power Rank.

Stanford’s schedule gets tougher this weekend with a game at USC. The Cardinal offensive line will get tested against a USC team that is much better than their 49th ranking in scoring defense. While much has been made of three senior linemen that graduated last year, only center Chase Beeler made an NFL training camp roster, according to Stanford’s media guide. Moreover, tackle Jonathan Martin and guard David DeCastro, the returning starters, are rated as the 9th and 26th best NFL prospects overall by ESPN. Center Sam Schwartzstein, guard David Yankey and tackle Cameron Fleming have filled the starting roles and powered the Stanford to 10.1 yards per carry (446 total) against Washington last week. The Power Rank predicts a 18.9 point win by Stanford, significantly higher than the line of 8 in favor of Stanford. USC has a 15% chance of pulling an upset.