The accuracy of The Power Rank’s 2014 college football predictions

How did my college football predictions do in 2014? Here, I look at not only my posted numbers for all games but also the forecasts made on this site and other outlets such as Grantland, Deadspin and Bleacher Report.

It’s 2015, and I’m making a full effort to track and report on all of my predictions. It started with baseball this spring, and it will continue through football and basketball.

Let’s get started.

Best prediction

Before the BCS title game, I wrote on Deadspin about how Ohio State presented a terrible match up for favorite Oregon. Ohio State had a vicious rushing attack that had just mauled a strong Alabama defense. Oregon had an average rush defense.

During the game, Ohio State RB Ezekiel Elliott gashed Oregon for 246 yards on 36 carries (6.8 yards per carry). Despite 4 turnovers, Ohio State won 42-20.

In the comments of the Deadspin article, a reader wrote this:

It is the start of the fourth, and it is creepy how on point your predictions are.

Two of my preseason predictions make honorable mention.

Auburn to regress

Auburn had a dream season in 2013, as they rose from the ashes of the SEC West to win the conference and play in the BCS championship game. However, they got the benefit of a few lucky plays (a tipped hail mary completion against Georgia, a field goal returned for a touchdown against Alabama).

In August, I wrote about how Auburn would have a tough 2014 season because of their schedule and the small chance they benefit from those lucky type plays again. Auburn fans didn’t like that I called them lucky.

Auburn couldn’t reproduce those plays in 2014. After a magical 12-2 season in 2013, they fell to 8-5 last season. Part of their demise was a tough cross division game at Georgia that they lost.

TCU to win the Big 12

My other favorite preseason prediction was that TCU would win the Big 12.

I actually went against my numbers on this one as Oklahoma had a higher win probability. However, no one gave TCU a chance, and I had them ranked 14th in the preseason.

TCU had a tremendous season as they went 12-1 and finished as co-champions with Baylor of the Big 12.

Worst prediction

Grantland asked me to predict the Heisman winner during the preseason. I don’t make player predictions, so I had some fun and picked Stanford QB Kevin Hogan.

I thought I had some strong reasoning, but Hogan came no where near the Heisman conversation. He led a Stanford offense that made red zone stalls a season long habit. This led to a disappointing 8-5 season.

Halfway through the season, Grantland gave me a do over and asked for another Heisman prediction. This time I go with Bo Wallace, the quarterback of a 6-0 Ole Miss team. Part of my reasoning was his improved completion percentage the first half of the season, an oh so huge sample size to make a judgment.

I wrote the following about my Wallace pick.

It’s hard to deny a blond quarterback from an unexpected SEC contender.

Then Wallace had a terrible second half of the season. He couldn’t make a play in a close game against LSU, and Ole Miss lost their first game of the season. The once mighty Rebels lose two more SEC games before getting blown out by TCU 42-3 in a bowl game.

Don’t ask me about the Heisman.

The Ohio State season end surge

For predictions based on my numbers, I was disappointed to not predict Ohio State’s surge at the end of the season. Before the Big Ten title game, they were 13th in my team rankings. Their loss to a bad Virginia Tech team at home pulled them down.

Then Ohio State plays the 3 best games any college football team has ever put together. They become the first national champion in the playoff era.

Member predictions

Members get access to my best predictions for spreads and totals.

The member predictions with the most value are the college football totals, which were posted from week 6 to the end of the season in 2014. These predictions went 53.3% against opening totals (273-239-4) and 51.5% against closing totals (260-245-9).

However, these numbers do not tell the entire story. When the predicted total differed from the opening total by more than 4 points, the market total moved in the direction of the prediction 90.4% of the time (122-13, with two totals that didn’t move).

On average, the final total moved 3.5 points in the direction of the model prediction. Some refer to this as closing line value.

For the entire 2014 season, spread predictions for members were 50.1% against the opening line (367-366-19) and 48.6% against the closing line (357-378-8). Modifications will be made to this model for 2015.

To learn more about becoming a member of The Power Rank, click here.

Public predictions

On the predictions page, I posted a margin of victory for each college football game.

These predictions got the game winner correct in 70.4% of games (539-227). It’s interesting that my preseason model, which doesn’t use data from the regular season, predicted a higher percentage of game winners (71.1% on 482-196 with no predictions on the other games).

Against the markets, the public predictions won at 49.9% against the opening line (370-371 with 18 pushes) and 48.5% against the closing line (363-385 with 9 pushes). It’s tough to beat the markets on every game.

The public predictions will be reworked this season. There’s room for great improvement, especially since these predictions were 53.8% against the opening spread through week 8 of the season.

Playoff probabilities on Bleacher Report

Playoff predictions on 11-18-2014.

On Bleacher Report, I predicted which teams would make the four team playoff based on the committee rankings. To learn more about these simulation methods, click here.

Overall, I thought the predictions did pretty well. Mississippi State was first in the committee’s first rankings. However, my numbers thought they wouldn’t make it due to tough games at Alabama and Ole Miss. Mississippi State lost both games and didn’t make the playoff.

By week 12 of the season, Alabama, Oregon and Florida State had the highest chance to make the playoffs by my calculations. Eventually, all 3 of these teams made the playoff.

However, the predictions were off the last week of the season as my numbers had TCU instead of Ohio State for the last spot. As I mentioned earlier, it was difficult to predict Ohio State’s surge at the end of the season based on their previous numbers.

However, my methods also need work. I had no way of knowing how the committee would value a conference championship. For 2015, I’ll account for this in my model.

Even with this improvement, there are still human factors out of my control. Though the committee placed an emphasis on a conference championship, Big 12 commissioner Bob Bowlsby still presented them with co-champions in Baylor and TCU. It might make more sense to crown Baylor the champion as they beat TCU.

My other decent predictions

Some of my other predictions had the right idea but didn’t nail it.

Thanks for reading.

How safe is Oregon from an upset against Ohio State?

cfb_playoff_trophyOregon looks like a clear favorite over Ohio State in the college football championship game.

The markets opened with Oregon as a 7 point favorite, which implies a 70% win probability. The predicted margin of victory is even higher with my college football team rankings based on margin of victory.

After an early loss to Arizona, Oregon has been dominant. Only UCLA has come within two touchdowns of beating Oregon. This stretch of games included a rematch against Arizona and the playoff semi-final against Florida State.

Ohio State barely made the college football playoff after an early loss to Virginia Tech, a team that went 3-5 in the ACC. They’re playing a third string quarterback lucky to have receivers talented enough to catch his jump balls.

Oregon should win, right?

In reality, Ohio State is a terrible match up for Oregon. Let me explain.

Oregon’s biggest match up problem

Ohio State has an elite ground game. To quantify this, let’s look an efficiency statistic: yards per carry. In college football, sacks count as rushes in the official statistics. Since sacks are pass plays, I exclude these plays in calculating yards per carry.

To adjust yards per carry for strength of schedule, I use a ranking algorithm I developed based on my research in statistical physics. While Ohio State has the 7th best raw yards per carry, these adjustments move them up to first. Ohio State is predicted to gain 6.78 yards per carry against an average FBS rush defense.

And Oregon has essentially an average rush defense (52nd of 128). Ohio State ran all over Wisconsin (13th ranked rush defense) and Alabama (2nd). They should do even better against Oregon.

The visual shows the difficult match up for Oregon. The blue dots represent Ohio State’s pass and rush offense. The smaller green dots show Oregon’s defense, and better defenses appear further to the right to facilitate comparisons. You’re looking at how a unit compares to average.

Ohio State's offense vs Oregon's defense

The gap between Ohio State’s rush offense and Oregon’s rush defense shows the clear advantage for the Buckeyes.

They will give most of the carries to Ezekiel Elliott, who has gained 6.9 yards per carry this season. We all saw his speed when he outran the Alabama defense for a 85 yard touchdown in the semi-final game. Quarterback Cardale Jones will also run the ball, and he’s a load to bring down at 6’5″, 250 pounds.

Offensive line coach Ed Warriner deserves much of the credit for Ohio State’s explosive run game. He had to groom four new starters this year, and none of the candidates had 5 star recruiting credentials. While the offensive line came into the season with question marks, it now looks like the strength that could carry them past Oregon.

Oregon’s other match up problem

Oregon, led by Heisman winning quarterback Marcus Mariota, excels at throwing the ball. To quantify this, let’s look at yards per pass attempt, an efficiency statistic that includes sacks. After adjusting for schedule, Oregon has the top ranked pass offense. They are predicted to throw for 9.04 yards per attempt against an FBS average pass defense.

However, Ohio State’s strength on defense is against the pass. They had the 9th best pass defense by adjusted yards per attempt. Against Alabama, they didn’t allow star receiver Amari Cooper to make big plays. While Cooper averaged 13.9 yards per catch this season, his longest against Ohio State was 15 yards.

The visual shows how Oregon’s offense matches up with Ohio State’s defense.

Oregon's offense vs Ohio State's defense

The gap between Oregon’s offense and Ohio State’s defense shows the size of the advantage. Oregon should still be able to throw the ball against Ohio State. However, it won’t be as easy as against Florida State.

The visual also show Oregon’s edge in running the ball. They should run it often (and they did on 55.9% of plays this season) and set up play action for Mariota.

Prediction

For college and pro football this season, I started aggregating many predictions into one ensemble prediction. This ensemble, which includes my adjusted numbers and data from the markets, predicts Oregon by 3.2 points, which corresponds to a 59.5% win probability.

However, you should never blindly trust numbers, especially in a game with mismatches. One of the predictors in the ensemble accounts for passing and rushing separately for each team. It considers Ohio State’s significant edge in running the ball and that Ohio State runs the ball on 59.3% of plays.

This matchup model predicts a 50-50 game between Ohio State and Oregon.

I think the game will be very close. Can Mariota have a monster game and carry his team? Or does Elliott break off big run after big run?

This game most likely comes down to a field goal in the final minutes. I give a slight edge to Oregon to win, but don’t be surprised if Ohio State pulls it out.

Annotated ensemble predictions for week 15 of college football

cfb_playoff_trophyIt’s championship week in college football, and the selection committee announces the four teams for the first college football playoff on Sunday. Will we see any upsets this weekend that alter the playoff landscape?

The numbers suggest we won’t see an upset with the top 3 teams (Alabama, Oregon and TCU). TCU doesn’t play in a championship game and hosts one of the weakest teams in their conference (Iowa State). Their 92% probability to make the playoff is higher than the chance for Alabama and Oregon.

However, we might see upsets for the next three teams: Florida State, Ohio State and Baylor. My calculations give a 81% chance that at least one of these teams loses this weekend.

These predictions come from an ensemble calculation that aggregates many different predictors. The predictions based on team rankings are only one factor in the ensemble.

The offense and defense rankings mentioned below are based on yards per play adjusted for strength of schedule. All predictions and rankings are available to members.

Oregon over Arizona by 14.7

I understand Arizona has won the last two games against Oregon. And it’s easy to just look at head to head games and think Arizona should win again.

However, you shouldn’t emphasize head to head too much. There is variance in sports. Teams don’t always play at the same level every game. A better team can lose to a worse team. Sports would be much less fun if this didn’t happen.

Arizona capitalized on a +3 turnover margin against Oregon in 2013. This season, Oregon tackle Jake Fisher didn’t play against Arizona, and the pass rush affected QB Marcus Mariota.

The past will not matter once these two teams kick off the Pac-12 championship game. Expect Oregon to win.

Alabama over Missouri by 14.1

Missouri has the 5th ranked defense, and their offense moves higher each week in my rankings (currently 59th). However, they will struggle against the best team in the nation.

TCU over Iowa State by 26.3

These numbers do not capture motivation. I think TCU has motivation to win big to ensure that committee keeps them in the top 4.

Florida State over Georgia Tech by 7.5

Expect a lot of points in this game, as Florida State and Georgia Tech have the 34th and 73rd ranked defense by yards per play adjusted for strength of schedule.

Wisconsin over Ohio State by 4.0

This prediction has been adjusted by 4 points for the injury to Ohio State QB JT Barrett. Ohio State also has issues on defense as their 50th ranked rush defense faces Wisconsin’s top ranked rush offense.

Baylor over Kansas State by 7.0

Baylor QB Bryce Petty will play against Kansas State after suffering concussion symptoms last week. While Baylor’s offense will find it difficult against Kansas State’s 13th ranked defense, they should win.

Boise State over Fresno State by 16.1

The divisions of Mountain West conference are as lopsided as Eastern and Western conference of the NBA. The 3 best teams (Boise State, Colorado State, Utah State) play in the Mountain Division. West Division champion Fresno State, ranked 85th in my team rankings, has a 12.8% chance to beat Boise State.

Predicting the college football playoff after week 13

Screen shot 2014-11-19 at 9.52.14 AMWhich teams will make the college football playoff? How certain are we about these probabilities?

Each week at Bleacher Report, I publish my playoff probabilities and write about the story behind the numbers. Last week, the article had over 159,000 visits and 459 comments.

This week, we explore the following story lines around the college football playoff.

  • How did Mississippi State’s odd increase after losing to Alabama?
  • What is the probability that a two loss team makes the playoff?
  • Can Ohio State climb from obscurity into the top 4 by season’s end? We evaluate their prospects in the Big Ten championship game.

To play with the visual and read the article, click here.

Interview with Armen Williams and Levack on 104.5 The Team

Armen Williams and Levack from 104.5 The Team (ESPN Radio in Albany, NY) interviewed me yesterday. We talk about college football, the playoff and all the big games for week 12 of the 2014 season.

It was a blast. On some interviews, I only talk with the hosts during the interview. Even though they were running late, Armen and Levack took time to chat with me before the interview.

And no one else has asked me whether I could beat Andrew Luck at Jeopardy.

You can also listen to the interview on Youtube.