Podcast: Wisdom of Crowds Prediction of the 2018 NFL Draft

On this episode of The Football Analytics Show, I use a wisdom of crowds approach and aggregate mock drafts to make accurate predictions for the 2018 NFL Draft.

Last year, this predictor was more accurate than all but one of the constituent mock drafts. In this episode, I discuss where the top quarterbacks go in the first round, who will take Saquon Barkley and possible improvements to the model.

You can check out the complete predictions for the 2018 NFL Draft by wisdom of crowds.

To listen on iTunes, click here.

To listen here, click on the right pointing triangle.

Predicting the 2018 NFL draft with wisdom of crowds

In 1906, Francis Galton went to a country fair which had a contest. 787 people paid a small fee to guess the weight of an ox. The people who made the best guesses won prizes.

Galton became interested in this crowd based estimate. Initially, he didn’t think taking an average guess would lead to an accurate estimate of weight. Galton didn’t have a high opinion of people, as he believed only certain people had the traits to keep societies healthy.

However, his curiosity got the best of him. Most likely by pencil and paper, he calculated the average guess of the crowd: the ox weighed 1197 pounds. The actual ox weighed 1198 pounds.

The wisdom of crowds is powerful. In sports, we see many examples of this, from the betting markets to the surprising accuracy of the preseason college basketball polls.

Here, we apply the wisdom of crowds to the 2018 NFL draft.

Aggregating data from mock drafts

A wisdom of crowds prediction for the NFL draft comes from combining the data from multiple mock drafts. For each mock draft, a player gets 32 points for being projected first, 31 points for second, and so on down to 1 point for the final pick of the first round.

Players get ranked based on total points, and this rank becames his predicted spot in the draft. In 2017, this wisdom of crowds model performed better than all but one of the constituent mock drafts.

The wisdom of crowds model isn’t perfect. It doesn’t consider the specific needs of a team, like a Denver franchise in need of a quarterback. I address this issue on a team by team basis in the predictions below.

However, the wisdom of crowds should be better than any mock draft at predicting a player’s relative draft position. For example, when will Louisville QB Lamar Jackson get picked compared to the 4 quarterbacks in front of him?

Let’s look at the wisdom of crowds predictions based on 25 mock drafts for 2018.

1. Cleveland, Sam Darnold, QB

After a harrowing year with Deshone Kizer at QB, the Browns will almost certainly take a QB with the top pick.

More mock drafts have Josh Allen at first (13) than Sam Darnold (11). However, Darnold didn’t slip below 5th in any mock draft, which makes him first by total points.

2. New York Giants, Saquon Barkley, RB

“This whole myth of devaluing running backs, I find it kind of comical.” said Giants GM Dave Gettleman recently. He hasn’t absorbed how data values passing significantly more than rushing in the NFL, which you can read about in my manifesto.

QB Eli Manning is old and not very good. However, 16 of 25 mock drafts have the Giants taking Barkley with the second pick.

3. New York Jets, Josh Allen, QB

In 2017, Josh Allen completed 56% of his passes compared to the 60% college football average. His Wyoming offense finished 127th out of 130 teams in my yards per play adjusted for strength of schedule. No matter where Allen gets drafted, I expect him to be the biggest QB bust since JaMarcus Russell.

This third spot is tricky one for wisdom of crowds. The 25 mock drafts all had the Jets taking a QB, but none of them predicted Allen. Baker Mayfield went to the Jets in 10 mock drafts, while Josh Rosen and Sam Darnold were taken in 9 and 6 mock drafts respectively.

4. Cleveland, Bradley Chubb, DE

Chubb has averaged 10 sacks and 22 tackles for loss the past two seasons at North Carolina State. He would be an excellent addition to any defense.

5. Denver, Quenton Nelson, G

Nelson was part of a Notre Dame offensive line that powered the second best rush offense by my yards per carry adjusted for schedule in 2017.

Denver may also go for a QB here, as projected starter Case Keenum might not be able to replicate his excellent 2017 season. 9 of 25 mock drafts had the Broncos taking Josh Allen at this spot.

6. Indianapolis, Baker Mayfield, QB

Five years from now, I expect Baker Mayfield to be the best QB from this draft. Over the past two seasons, he led Oklahoma to the best offense by my adjusted yards per play by a wide margin.

However, Indianapolis has Andrew Luck, so they won’t take Mayfield at this slot. Nelson (11) and Chubb (8) come up most for the Colts with the 6th pick.

7. Tampa Bay, Josh Rosen, QB

Rosen completed 63% of his passes in 2017 in leading the 12th ranked pass offense by my adjusted yards per attempt. In addition, he started every game for UCLA that he wasn’t hurt since the beginning of his freshman year. This gives me some confidence he can play in the NFL, much more so than Allen.

With Jameis Winston, Tampa Bay doesn’t need a QB. Derwin James (10) and Minkah Fitzpatrick (8) show up most at this spot in the mock drafts.

8. Chicago, Roquan Smith, LB

Smith accumulated 6.5 sacks and 14 tackles for loss on a Georgia defense that excelled in 2017. The Bulldogs ranked 3rd in my adjusted yards per play.

9. San Francisco, Denzel Ward, CB

5 cornerbacks were taken in the first round of last year’s draft. Only two are projected for this season, and the wisdom of crowds likes Ward the best.

San Francisco signed CB Richard Sherman this off season, so Roquan Smith and Tremaine Edmunds are other possibilities here.

10. Oakland, Minkah Fitzpatrick, S

Oakland’s pass defense has ranked 29th and 32nd the past two seasons in my adjusted yards per attempt. They can use any and all help, so expected Fitzpatrick or Ward to go here.

11. Miami, Tremaine Edmunds, LB

Edmunds had 10 sacks and 30.5 tackles for loss the past two seasons at Virginia Tech. Miami also might take a QB here if Mayfield or Rosen fall this far.

12. Buffalo, Derwin James, S

Florida State had an awful season after QB Deondre Francois went down. However, the defense was solid at 6th in my adjusted yards per play, and James played a key role.

Buffalo may also take a QB here if Rosen or Mayfield drop this far.

13. Washington, Vita Vea, DT

Vea is 6’5″, 340 pounds. That should help the Redskins defense, and 11 of 25 mock drafts have Vea in this spot.

14. Green Bay, Marcus Davenport, DE

At UTSA, Davenport had 8.5 sacks and 17.5 tackles for loss. The Packers need help on defense, so Davenport makes sense. However, just as many mock drafts had CB Joshua Jackson (6) at this spot as Davenport.

15. Arizona, Calvin Ridley, WR

Ridley caught fewer balls in each successive season at Alabama. However, he did average 15.3 yards per reception in 2017. While 6 mock drafts had Ridley to the Cardinals, the same number had QB Lamar Jackson at this spot.

16. Baltimore, Mike McGlinchey, OT

McGlinchey was part of a Notre Dame offensive line that powered the second best rush offense by my adjusted yards per carry in 2017. Baltimore might also go for Calvin Ridley here, as Joe Flacco led the worst pass offense by my adjusted yards per attempt in 2017.

17. Los Angeles Chargers, Joshua Jackson, CB

The Chargers had a surprisingly good pass defense in 2017 (5th in my adjusted yards per attempt), so they probably don’t take a cornerback. The mock drafts think the Chargers need help at defensive tackle, as 10 and 7 drafts picked Da’Ron Payne and Vita Vea respectively.

18. Seattle, Da’Ron Payne, DT

Alabama’s defense ranked 1st in my adjusted yards per play in the 3 years with Payne as the anchor on the line. He’s smaller than Vea (305 pounds), but still looks like a solid first round pick.

In the 25 mock drafts, 13 different players were picked here. Seattle may settle on any of a number of players at this 18th spot.

19. Dallas, Lamar Jackson, QB

Jackson increased his completion percentage the past two season, but it topped out at 59% in 2017 (60% college football average). We’ll see if he continues to develop in the NFL.

Dallas has their QB, at least for now, in Dak Prescott. The mock drafts like the Cowboys to pick a wide receiver at 19th. The primary candidates are DJ Moore (7), Courtland Sutton (7) and Calvin Ridley (6) if he drops this far.

20. Detroit, Harold Landry, DE

Landry posted an outstanding 16.5 sacks and 22 tackles for loss as a junior in 2016. He numbers slipped last season because of an ankle injury, but he could help the pass rush of the Lions.

8 of 25 mock drafts had Landry in this spot, well ahead of Davenport at 4.

21. Cincinnati, DJ Moore, WR

The Bengals need help on the offensive line. 9 mock drafts had Iowa’s James Daniels here, and another 11 had other offensive linemen.

22. Buffalo, Leighton Vander Esch, LB

Vander Esch had 141 tackles on a solid Boise State defense that ranked 21st in my adjusted yards per play. He seems like a good fit for Buffalo as 6 mock drafts picked him here.

23. New England, James Daniels, C

Daniels might not be here as Cincinnati picks two spots before New England. And it’s amusing that the 25 mock drafts has 12 different players in this spot, none of which were Daniels.

24. Carolina, Rashaan Evans, LB

Can’t go wrong with a Alabama player on defense, right? Evans had 6 sacks and an additional 7 tackles for loss in 2017.

The mocks drafts had 14 different players at this spot, none of which were Evans. Carolina won’t take a QB, but otherwise expect a surprise at 24th.

25. Tennessee, Courtland Sutton, WR

The mock drafts like the Titans to pick a pass rusher here. Harold Landry (6) and Sam Hubbard (5), a DE from Ohio State, had the most votes.

26. Atlanta, Will Hernandez, G

On a recent episode of The Football Analytics Show, NFL Network analytics expert Cynthia Frelund said to watch out for Hernandez, the lineman from an awful UTEP team. However, Atlanta most likely goes after Taven Bryan to bolster their defensive line (15 of 25 mock drafts).

27. New Orleans, Jaire Alexander, CB

The mock draft think the Saints draft a tight end. Hayden Hurst from South Carolina (7), Dallas Goedert from South Dakota State (5) and Mike Gesicki from Penn State (2) are the most likely picks.

28. Pittsburgh, Taven Bryan, DT

Bryan most likely goes to Atlanta two spots before Pittsburgh. But in this model, he slips to 28th since he’s not a first rounder in 5 of 25 mock drafts.

10 mock drafts like the Steelers to take Rashaan Evans, as they have a depleted linebacking corp without Ryan Shazier.

29. Jacksonville, Isaiah Wynn, G

In 2016, Georgia struggled under first year head coach Kirby Smart. They ranked 44th in rush offense by my adjusted yards per carry.

Something clicked in 2017, and Georgia shot up to 7th in rush offense. Wynn now most likely goes in the first round.

30. Minnesota, Kolton Miller, OT

The Vikings just spent a ton of money on QB Kirk Cousins, so it makes sense to protect that investment with a lineman. Wynn and Hernandez are also candidates here if they drop this far.

31. New England, Mike Hughes, CB

The Patriots could use a cornerback with the strange benching in the Super Bowl and subsequent departure of Malcolm Butler. 10 mock drafts also have tackle Kolton Miller here.

32. Philadelphia, Connor Williams, OT

The 25 mock drafts had 16 different players here. The defending Super Bowl champions take the best player available.

Evaluating a wisdom of crowds prediction for the 2017 NFL draft

Last season, I made a wisdom of crowds predictor for the 2017 NFL by combining the predictions of 15 different mock drafts.

If a mock draft predicted a player to go first, the player earned 32 points. If a mock draft predicted a player to go second, the player earned 31 points, and so on down to 1 point for the last pick of the first round.

The players got ranked by points to make the wisdom of crowds prediction. You can get the full list of predictions here.

How did the wisdom of crowds predictor do compared to the individual mock drafts?

Error for each NFL draft prediction

To evaluate each draft predictor, I evaluated the distance from predicted to actual draft spot for each player. For example, Mitch Trubisky went 2nd in the draft, so the wisdom of crowds prediction of 8th has a distance of 6. Jonathan Allen went 17th despite a prediction of 3rd, which gives a still positive distance of 14. The total distance for all 32 predicted players gives an error for a draft prediction.

I tracked each player compared to his actual draft slot for the first two rounds. In a few cases, a predicted player got drafted outside these top two rounds. To evaluate the distance, I assigned draft slot of 65 (first pick of the third round), plus a penalty of 3. Only 7 of the mock drafts had a prediction outside the first two rounds, and they are denoted by a * in the list below.

Here are how the 15 mock drafts stacked up compared to wisdom of crowds.

1. Sporting News, 222
2. The Power Rank Wisdom of Crowds, 239
3. Todd McShay, 249
4. CBS, 259*
5. Daniel Jeremiah, 263
6. Mel Kiper, 265
7. Drafttek, 266*
8. Lance Zierlein, 277
9. Charlie Campbell, 290
10. Sports Illustrated, 291*
11. SB Nation, 295
12. Fox, 306
13. Newsday, 310*
14. Cris Collinsworth, 344*
15. Walter Cherepinsky, 405*
16. Pro Football Focus, 431*

I took this data on April 19, 2017. The predictions may have changed for some of these mock drafts as authors made updates before the first round of the NFL draft on April 27, 2017.

The wisdom of crowds predictor did better than any constituent mock draft except for one: Vinnie Iyer at Sporting News.

In future years, I plan to use the full draft to evaluate the mock drafts, not only the first two rounds with an arbitrary penalty for picking a player outside of these rounds. In most cases, this better method would make the error larger for the mock drafts with a *.

Podcast: Chase Stuart on the analytics of NFL draft picks

On this pre-draft episode, Chase Stuart, founder of Football Perspective and 2017 panelist at the Sloan Sports Analytics Conference, joins me to talk about the NFL draft. Among other topics, we discuss the following:

  • The story behind his ubiquitous NFL draft trade value calculator
  • How to use Approximate Value to access every player, even offensive linemen and defensive backs
  • The reason an NFL GM might use the Jimmy Johnson draft value chart instead of his analytics based chart
  • The surprising admission of the Patriots on why they traded for Brandin Cooks
  • Why Chase writes every day at his site Football Perspective

After the interview, I talk about 3 highlights of my wisdom of crowds model for predicting the 2017 NFL Draft.

To listen on iTunes, click here.

To listen on the site, click on the right pointing triangle.

Predicting the 2017 NFL Draft by Wisdom of Crowds

You want know which player your team will pick in the 2017 NFL draft.

You’re not alone. Hope springs eternal in the off season, even in Cleveland. There’s no shortage of mock drafts on the internet to satisfy this interest.

As a football analytics site, The Power Rank wants to use this data to make even better predictions on the draft. I take a wisdom of crowds approach and combine the predictions of 15 different mock drafts.

Research suggests this wisdom of crowds will lead to better draft predictions. A study from 2014 found that this approach better predicted draft position than any individual mock draft.

The preseason AP college football poll also uses the wisdom of crowds. My work has found that the higher ranked team wins almost 60% of bowl games, a remarkable rate because this predictor uses no data from the regular season.

For the NFL draft, the wisdom of crowds approach has clear flaws. Teams use the draft to fill certain needs, so some of these predictions will not make sense. I’ll discuss these issues based on my NFL adjusted statistics below.

However, these predictions should capture the relative position of where each player gets picked in the draft. Let’s look at the wisdom of crowds prediction for the 2017 NFL draft.

1. Cleveland, Myles Garrett, DE

The Browns needs lots of help of both sides of the ball. Garrett, the pass rush phenom from Texas A&M, was first in every mock draft.

2. San Francisco, Solomon Thomas, DE

When you need as much help as the Niners on both sides of the ball, it makes sense to get a defensive lineman that can rush the passer. Stanford stud Thomas stays in the Bay Area to join the Niners.

3. Chicago, Jonathan Allen, DE/DT

The Bears had the 11th best pass offense by adjusted yards per attempt last season despite a rash of QB injuries. Hence, it makes sense to draft a defensive player like Allen, the defensive end from Alabama with 10.5 sacks and 16 tackles for loss last season.

4. Jacksonville, Marshon Lattimore, CB

The Jags had the 6th best pass defense by my adjusted yards per attempt, perhaps a surprise. Still, this is a strong draft at cornerback, one of the NFL’s toughest positions to fill. Lattimore, the cornerback from Ohio State, emerges from the wisdom of crowds as the top secondary player.

The pass offense was terrible by my adjusted yards per attempt (27th). However, it doesn’t seem like this team is ready to give up on young QB Blake Bortles.

5. Tennessee, Malik Hooker, S

The Titans got this pick by trading away the last year’s top pick, which the Rams used on QB Jared Goff. The Titans need help on defense, and either Hooker or Lattimore, Ohio State teammates from last season, makes sense here.

6. New York Jets, Jamal Adams, S

By my numbers, the Jets need help everywhere except run defense. They should pick the LSU safety Adams unless they want to play dice with a quarterback at 6th.

7. Los Angeles Chargers, Leonard Fournette, RB

The Chargers can’t run the ball, as they ranked 27th in my adjusted yards per carry last season. Fournette could help an offense that can throw the ball with Phillip Rivers.

8. Carolina, Mitch Trubisky, QB

There’s no way the Panthers take Trubisky with Cam Newton locked in as their franchise QB.

But this begs the question of why any team would take Trubisky in the first round. He only started one season at North Carolina, which means he couldn’t beat Marquise Williams out for the job the prior two seasons. Williams isn’t currently signed by any NFL team.

Kevin Cole did a nice Bayesian analysis to compare Trubisky with Deshaun Watson. The small sample size on Trubisky makes it hard to be certain of his pass efficiency.

Back to Carolina, who had a surprisingly good pass defense (10th by adjusted yards per attempt) after letting Josh Norman go and playing young cornerbacks. A running back like Fournette or McCaffrey makes sense here to help out Newton.

9. Cincinnati, OJ Howard, TE

By last season’s numbers, the Bengals need help with pass rush and pass protection. However, Alabama TE Howard would add to an explosive offense that already has QB Andy Dalton and WR A.J. Green.

Also, Cincinnati ended up 11th in my member team rankings despite a 6-9 record. Get this pick right, and they can easily bounce up to the playoffs in 2017.

10. Buffalo, Mike Williams, WR

The Bills ranked 25th by my adjusted yards per pass attempt last year, so a receiver makes sense here. Williams outplayed CB Marlon Humphrey when his Clemson Tigers beat Alabama in the college football championship game.

11. New Orleans, Reuben Foster, LB

The Saints need help on defense, so a linebacker who had 13 tackles for loss for Alabama last season makes sense here.

Foster did have a positive drug test at the NFL combine for dilute urine. If Foster drops, pass rusher Derek Barnett is another good pick for the Saints.

12. Cleveland, Christian McCaffrey, RB

Rumors suggest that the Browns like QB Mitch Trubisky, and this model suggests they would have to trade up from 12th to draft him.

13. Arizona, Derek Barnett, DE

Carson Palmer was awful for the Cardinals last season, as they dropped from 1st in 2015 to 22nd in 2016 in my adjusted yards per pass attempt.

However, a lot of the mock drafts have Arizona drafting a defensive player. A pass rush specialist that Barnett who had 13 sacks and 19 tackles for loss will help any defense.

14. Philadelphia, Corey Davis, WR

The Eagles pass offense ranked only 28th in my pass offense numbers, as they didn’t let rookie QB Carson Wentz throw deep that much. A talent like Davis from Western Michigan should help the pass offense, although they did sign Alshon Jeffrey and Torrey Smith in free agency (ht G.Moore).

15. Indianapolis, Haason Reddick, LB

The Colts had all kinds of problems on defense last season. Reddick had 9.5 sacks and 22.5 tackles for loss at Temple last season.

16. Baltimore, John Ross, WR

The Ravens won the Super Bowl right before QB Joe Flacco’s contract ended. With this recency bias, they now overpay him and have the 26th ranked pass offense to show for it.

Still, the Ravens contend for the playoffs every year, and they’ve built this team through the draft. Ross, the speedy receiver from Washington, should help their pass offense.

17. Washington, Deshaun Watson, QB

The Redskins still have Kirk Cousins at QB, although his long term contract status is uncertain. They could draft Watson, the accurate QB from Clemson, with an eye on the future.

Washington also had an awful secondary last season despite having Josh Norman at cornerback. Drafting Gareon Conley, the cornerback from Ohio State, probably makes the most sense.

18. Tennessee, Gareon Conley, CB

With the 22nd ranked pass defense, drafting a cornerback here for the Titans is a no brainer.

19. Tampa Bay, Cam Robinson, OT

The Bucs had the 28th ranked rush offense of 32 teams in my adjusted yards per carry. Robinson, the tackle out of Alabama, might help this situation, although they could also draft a running back like Dalvin Cook here.

20. Denver, Taco Charlton, DE

Charlton makes no sense as the Broncos have an incredible pass rush but awful pass protection. Cam Robinson out of Alabama or Ryan Ramczyk from Wisconsin would help bolster the offensive line for whoever plays QB next season.

21. Detroit, Dalvin Cook, RB

The Lions need help in the secondary to help CB Darius Slay. After Lattimore and Conley, the wisdom of crowds model like Marlon Humphrey from Alabama and Kevin King from Washington as the next best cornerbacks.

The Lions should also consider Taco Charlton, the defensive end from Michigan that blossomed with 10 sacks and 13 tackles for loss as a senior.

22. Miami, Ryan Ramczyk, OT

The Dolphins could use help with pass protection, as they’ve ranked 25th the past two years in my adjusted sack rate allowed. Ramczyk is a good choice.

23. New York Giants, Pat Mahomes, QB

The model predicts Mahomes as the third QB off the board, and 13 of 15 mock drafts had him as a first round selection. As Eli Manning is 36, the Giants would be wise to look for his replacement here.

24. Oakland, Marlon Humphrey, CB

The Raiders had a league worst pass defense by my adjusted yards per attempt. Humphrey out of Alabama makes perfect sense here.

25. Houston, Forrest Lamp, OG

The Texans need a quarterback, and they should hope that Watson or Mahomes drops to them at 25th. A offensive lineman like Lamp could help a run offense that ranked 24th by my adjusted yards per carry.

26. Seattle, Garett Bolles, OT

The Seahawks need lots of help on the offensive line, as they played an offensive tackle that didn’t start playing football until 2015. Bolles would help that offensive line protect Russell Wilson.

27. Kansas City, Kevin King, CB

Andy Reid likes to draft linemen, so don’t be surprised if the Chiefs draft an offensive line prospect like Bolles or Lamp. It could also make sense to draft a cornerback like King from Washington.

28. Dallas, Charles Harris, DE

The Cowboys would bolster their pass rush with Harris, the defensive end out of Missouri with 9 sacks and 12 tackles for loss last season. A cornerback choice like Kevin King also makes sense.

29. Green Bay, Malik McDowell, DT/DE

I’m not sure how accurate a wisdom of crowds model can predict the draft slot for McDowell. He has clear NFL talent but often looked disinterested at Michigan State last season. He made the first round in only 7 of the 15 mock drafts, but was selected as high as 17 in two of them.

If the Packers want to take a risk, they could draft McDowell to replace the departed Julius Peppers.

30. Pittsburgh, Tre’Davious White, CB

The Steelers could enhance a pass defense ranked 13th last season with White. A pass rusher like Charles Harris or Malik McDowell also makes sense here.

31. Atlanta, Jarrad Davis, LB

The Falcons should improve their pass rush, which ranked 23rd in my adjusted sack rate last season. Davis only had 6 tackles for loss in 9 games for Florida last season.

A defensive end like Takkarist McKinley from UCLA makes more sense here. McKinley was ranked 33rd by the wisdom of crowds model.

32. New Orleans, Jabrill Peppers, S

The mock drafts couldn’t make up their mind on Peppers. He was a first found pick in 8 of 15 drafts, but he went has high as 10th in one of them.

Peppers could really enhance the Saints defense with this explosiveness (13 tackles for loss with Michigan last season). Or he could struggle to see the field because of his inability to cover slot receivers in the NFL.