Who else wants to predict the 2013 college football season with confidence?

How good is your team this year?

Is your team the best in its division or conference?

Will your team win the rivalry game this year?

There is no shortage of media outlets that will attempt to answer these questions before a game has been played. At my local Barnes and Nobles this August, there were 6 different preseason college football magazines, including Athlon, Lindy’s and Phil Steele.

The preseason Coaches and AP poll will also give you their opinion on your team.

These established brands have been around since the dinosaurs. They have years of experience in making preseason rankings.

So why would you look at another preseason rankings, especially from an upstart like me? Who cares about a regression model using past team performance, returning starters and turnovers when you could just lean on these established brands?

tpr_preseason_accuracy

To put this in perspective, consider these results from The Prediction Tracker, a site that tracks the accuracy of predictions systems.

Since 2005, the famous Sagarin Predictor rankings have picked the winner in 61.9% of bowl games. The opening line from Vegas odds makers predicts 59.9%. Both Sagarin and the odds makers can consider games during the season, information not used in The Power Rank preseason rankings.

For the entire season, Sagarin Predictor has picked the winner in 73.6% of games. The Prediction Tracker has data on 35 systems for each season since 2005. The season accuracies range from 74.5% to 68.1%. Again, these systems can use games from the season, information not used in my preseason rankings.

To check out The Power Rank’s preseason rankings for 2013, click here.

Let’s look at two factors in calculating this accuracy.

Accuracy in past seasons

While I developed this regression model for preseason rankings during the summer of 2013, I went back and calculated the results during previous seasons.

This requires some care in only using data from previous seasons in the calculation. For example, the 2005 preseason rankings only uses data from the 2004 season and before.

A rating for each team

In calculating the predictive accuracy, I assume no home field advantage in a bowl game. Hence, the higher ranked team is predicted to win, which was accurate in 60.6% of bowl games.

However, home field advantage complicates a prediction for regular season games. Fortunately, The Power Rank’s preseason predictions assign each team a rating, which gives an expected margin of victory against an average FBS team.

To make a prediction on a regular season home game, three points are added to the home team before picking the team with the higher rating to win. This system predicted 70.5% of college football game winners over the entire season.

Uncertainty in any preseason rankings

These preseason rankings still contain a great amount of uncertainty. They do not consider a single game during the regular season. Even though these preseason rankings predicting over 70% of game winners the last 8 years, a team’s rank and rating can change drastically by the end of the season.

Let’s quantify this. These preseason rankings are based on a regression model that considers team history over the previous 4 years, returning starters and turnover margin. These predictors are used to fit the final team ratings for the next year. Hence, it makes sense to ask how much the predictions deviate from the actual ratings calculated from games that season.

In the eight seasons since 2005, 72.2% of teams have finished within 7 points of their predicted rating.

Seven points in a team’s rating can make or break its season. For example, Nebraska, 25th in the preseason rankings, moves up to 6th ahead of Georgia with an additional 7 points. However, they drop to 58th behind UCF if my estimate is 7 points too high.

Let’s look at the 3 factors that go into the regression model for these preseason rankings.

Team History

College football programs are large, lumbering institutions that do not change overnight. Alabama will always have a rich tradition that started with Bear Bryant and continues through Nick Saban. Nothing Rice can do will give my Owls the tradition of the Crimson Tide. Fans will trickle into Rice Stadium while Alabama will sell out the 100,000 plus seats in Bryant Denny.

Because of this persistence, the regression model uses a team’s rating in The Power Rank from the last 4 seasons to predict next season. From studying previous seasons, these ratings have an overwhelming effect on how a team will perform next season. Those 3 national championships that Alabama has won over the past 4 seasons matter, and this history gives the Crimson Tide their #1 ranking.

Returning Starters

A team greatly benefits from having players with past experience on the field. For example, Stanford only finished 9th in last year’s rankings despite their Rose Bowl win. However, the Cardinal have 15 starters returning, including many from a stellar defense. With the continual development of QB Kevin Hogan, who started the last 6 games of the season, Stanford should improve from last season’s results.

Turnovers

Big plays like fumble recoveries have an enormous impact on games. However, predicting future turnovers is as difficult as telling whether a coin will land heads or tails. It’s a counter intuitive result, as most fans believe in the big play defense that forces turnovers. But turnovers forced regress strongly to the mean.

The Oklahoma State defense discovered this regression last season. In 2011, the Cowboys forced 44 turnovers on their way to a 12-1 season and Fiesta Bowl win. Articles were written about how they practice forcing turnovers. In 2012, the Cowboys forced 22 turnovers, very close to the FBS average. They finished 8-5 last season.

While a positive turnover margin (take aways minus give aways) will enhance a team’s rating one year, this luck will most likely disappear the next season. This doesn’t always happen, as Rich Rodriguez found out during his 3 year tenure at Michigan. His teams never did better than a -10 turnover margin. However, the math is strong enough to include this factor in the regression model.

To check out the preseason rankings for 2013, click here.

Let’s look at two teams that jump out of these rankings.

Ohio State

With their unblemished 12-0 record from last season, most fans and pundits view the Buckeyes as the team to end the SEC’s run of seven national titles. However, my numbers think different, placing Ohio State 13th. They ended last season 14th, as the Buckeyes had a difficult time beating poor teams like UAB and Indiana. Top ranked teams like Alabama punished these types of opponents.

To enhance this comparison of Ohio State and Alabama, consider margin of victory. Last season, Ohio State had an average margin of victory of 14 points. This was about half of the 28 points that Alabama posted despite their loss to Texas A&M. In addition, Ohio State played a weaker Big Ten schedule than Alabama’s in the SEC.

I think 13th is a little low for Ohio State, particularly with the talent that coach Urban Meyer has recruited over the last two seasons. However, it’s unlikely Ohio State makes the jump to national title contender.

USC

A year ago, a USC was the media darling of the college football. The Trojans finished the previous season strong, including a dramatic win over Oregon that ended the Ducks’ national title hopes. Then QB Matt Barkley turned down the riches of the NFL to return for his senior season, which propelled the Trojans to #1 in the preseason AP poll.`

USC ended the 2012 season 7-6, far from the title contention that most expected for the Trojans. They lost a number of close games to good teams like Stanford, Oregon and Notre Dame. Then the wheels fell off when Barkley got hurt and Georgia Tech dismantled them in the Sun Bowl. This sent expectations for 2013 into the toilet, as some rankings do not even have USC as a top 25 team.

Of course, this is just as ridiculous as their #1 rank from last season. USC has placed in the top 20 of my rankings the last 3 seasons. Even with their 7-6 record last year, the Trojans ended the year 20th. This team history matters, and my preseason rankings have USC at 10th.

Accuracy in predicting conference winners

I don’t know anyone else that has checked the accuracy of preseason rankings in predicting game winners in college football. If you know of any studies, please let me know in the comments.

However, Stassen has been tracking the accuracy of how preseason magazines predict the final standings in division and conferences. Their site is definitely worth checking out.

My rankings do generate the winners of conferences through a Monte Carlo simulation. I wrote about this previously when complaining about the injustice of schedule in college football. However, I have not yet checked the accuracy of these results against the data of Stassen. It will probably have to wait until next season.

Thanks for reading. And if you still haven’t checked out the preseason rankings for 2013, here is that link one last time. 🙂

Comments

  1. MIke Wassenmiller says:

    Hi Ed, If I could, I’d really like your opinion on something. I thought about it when I was reading your thoughts about turnovers and how they fit into your regression model. My question is, do you value Off/Def pass efficiency and will you explain how it’s factors into your regression model?

    If you look at the National Champions over the last 13 years, they consistently rank very high statistically in offensive pass efficiency and especially so in defensive pass efficiency. I would add that these same National Champions were not as highly rated Nationally when it comes to turnovers – lost or gain – and in most cases ranked fairly pedestrian.

    With that said, back to my question…. Do you value Off/Def pass efficiency and will you explain how it’s factors into your regression model?

    Thanks for the great work you do. I enjoy being a member of your site.

    • Mike,

      Thanks for the thoughtful question.

      Offensive and defensive pass efficiency are factored into the model in the following way. Teams with good pass offense and defense (or a high differential of yards gained per play minus yards allowed per play) tend to have a better point differential. Teams with a better point differential tend to be more highly rated.

      So the link between passing and the preseason rankings is indirect. I think the preseason rankings would be better if the relationship were more explicit. Definitely on my to do list for next year.

      Again great question.

  2. Heya i am for the primary time here. I found this board and I in finding It truly useful & it helped me out a lot.
    I am hoping to provide something back and help others like you helped me.

  3. Its like you read my mind! You appear to know a lot about this, like you wrote the book in it or something.
    I think that you could do with a few pics to drive the message
    home a little bit, but other than that, this is great blog.
    A fantastic read. I’ll definitely be back.

  4. Hmm it appears like your website ate my first comment (it was super long) so
    I guess I’ll just sum it up what I had written
    and say, I’m thoroughly enjoying your blog. I too am an aspiring blog blogger but I’m still new to everything.
    Do you have any points for beginner blog writers? I’d definitely appreciate it.

  5. Re: “I don’t know anyone else that has checked the accuracy of preseason rankings in predicting game winners in college football. If you know of any studies, please let me know in the comments.”

    Check out John Wobus’ CF Ranking Analysis at http://sports.vaporia.com/fb-ranking-analysis.html
    The rank and score for the preseason predictions of each system tracked by Kenneth Massey’s CF Ranking Comparison is given in the last column (Week0) of each table.

    I am visiting your web site for the first time and enjoying the articles I have read so far.

  6. Can you help me u understand how Vegas uses data and sets lines in a favorable position?

    • I’m not the expert on that. I think Vegas sets a number to get half the bets on each side, but people have told me it’s more complicated than that.

Trackbacks

  1. […] Feng of ThePowerRank.com, one of my favorite sites, wrote an interesting post on the accuracy of college football preseason predictions. At the Power Rank his system has correctly predicted the winner in 70.5 percent of FBS games since […]

  2. […] In college football, team strength tends to persist from year to year. This makes it possible to use previous seasons to predict the current season. […]

  3. […] In college football, team strength tends to persist from year to year. This makes it possible to use previous seasons to predict the current season. […]

Speak Your Mind

*