Study: Trading up in the draft is a sucker's bet

[Skyy God]

I'm crediting blind luck rather than Pioli's savvy for the success of the Baldwin/Houston trade.

Today the Philadelphia Inquirer profiles Cade Massey, a professor at Penn's Wharton School of business. Already with a study under his belt arguing that the conventional wisdom of the Draft Value Chart is all wrong, Massey was contracted by an unnamed NFL team to study the history of the draft for market inequalities. He discovered something that won't come as a surprise to football fans: the draft is kind of a crap shoot.

There is skill in making individual picks, Massey says, but the fact that draft success isn't sustainable points to the conclusion that every team is fairly evenly matched. What seem to be indicators that drafting is a talent, like the Lions' drought or the Patriots' boom of a decade or so ago, are statistically expected aberrations.

"Some teams have great years, other teams have bad years - and it matters," Massey said. "But those differences aren't persistent year-to-year, which tells me that they are chance driven. Something between 95 and 100 percent - I'm not exaggerating - of team differences in the draft is driven by chance."

If you take issue with that, you'll have to math it out with the math guy; I'm just passing things along. But Massey's field of study seems perfectly designed to tackle the NFL draft— according to his site, his expertise in psychology and economics hones in on "judgment under uncertainty, with a focus on optimism, overconfidence, and learning."

Last year, he co-authored a study with the University of Chicago's Richard Thaler. Entitled "Loser's Curse: Overconfidence vs. Market Efficiency in the NFL Draft," it aimed to determine whether there are any patterns in how front offices value draft picks, and if those patterns expose an opportunity for greater value.

The study is embedded below, but here's a talking point: teams overvalue higher picks in part because they overvalue their own judgment in evaluating players, and that "overconfidence is exacerbated by information." The more front offices know about a prospect, the more they think they know, and they also assume other teams value that prospect as highly, creating a feedback loop that pushes players higher up the draft board than they may deserve.

In terms of practical takeaways, the study says to toss out that hoary chestnut, the Draft Value Chart. You know the one: teams have consulted it for decades to put a rough value on any trade including multiple picks. (Example: the second overall pick would be exactly equal to a sixth and a 16th.)

Massey says the chart is wrong, because it doesn't take into account salaries in a salary cap league. His study looks at "surplus value," or what a player actually gives you compared to what would be expected for his contract worth. Because rookie contracts keep salaries artificially low, the surplus values of draft picks are nearly always positive. But some are more valuable than others.

That treasured first pick in the draft is, according to this analysis, actually the least valuable pick in the first round! To be clear, the player taken with the first pick does have the highest expected performance, but he also has the highest salary, and in terms of performance per dollar, is less valuable than most players taken in the second round.
Here's the chart showing the surplus value of picks from drafts between 1994-2008 (when, it must be noted, the lack of a draft slotting system severely inflated the top picks' contracts). According to the research, teams gained more value from drafting late in the first round than they did early in the first.

Massey and Thaler have advice for GMs: only suckers trade up. Over those 14 years of drafts, they calculated the outcomes of every possible 2-for-1 trade for a first rounder using the Draft Value Chart, and found "overwhelming evidence that a team would do better in the draft by trading down." The team that would have traded down would have gained an average of 5.4 man-starts per season, with roughly the same amount of Pro Bowl appearances, at a cheaper cost.

The study was naturally controversial, in part due to misreadings. It deals only in probabilities, not in individual picks. If you trade up and land a player who turns out to be a superstar, it was a good trade. The study merely says that in most cases, that possibility doesn't justify the risk.

It also has little chance of making an impact in actual front office behavior, because of the very psychological barriers it cites. In terms of "impact"—tickets sold, media coverage garnered, general excitement—a top pick is always going to be desirable. Even more important is the corollary: the fear of missing out on a superstar. That's the kind of thing that costs GMs their jobs, and leads to moves that look sexy in the short term but don't work out over time.

http://deadspin.com/study-nfl-teams-...the-1378701238

[Jakemall]

Originally Posted by Frosty

http://nfltrades.tumblr.com/post/212...ywilliamstrade

Lavarr Arlington and Champ Bailey... You'd think they'd have done better with an entire draft.

[Tombstone RJ]

Originally Posted by patteeu

It was obviously the right pick, but it's not obvious whether it was lucky or a result of superior drafting skill. Same with the Luck pick, which was made by an entirely different regime than the Manning pick, btw.

I disagree because the Colts and Bill Polian had the choice to either draft Manning or Ryan Leaf and IIRC it was not an easy, shoe-in decision. Lots of pundits and draft experts were hyping the potential of Leaf and the limitations and problems of Manning (Manning had a much lower ceiling, not a strong armed QB, etc.).

So, there was absolutely a huge and conscious decision to take Manning over Leaf so it was indeed superior drafting. That was the Colts front office saying "we are taking Manning, even IF Leaf has more upside."

As for Luck and RG3, there was not nearly as big of a question about taking Luck over RG3 but there was still lots of people who said that RG3, much like Leaf, had more long term "upside" and potential to develop and blossom into a phenom later down the road. However, pretty much everyone agreed that Luck was a "can't miss" pick.

[patteeu]

Originally Posted by Tombstone RJ

I disagree because the Colts and Bill Polian had the choice to either draft Manning or Ryan Leaf and IIRC it was not an easy, shoe-in decision. Lots of pundits and draft experts were hyping the potential of Leaf and the limitations and problems of Manning (Manning had a much lower ceiling, not a strong armed QB, etc.).

So, there was absolutely a huge and conscious decision to take Manning over Leaf so it was indeed superior drafting. That was the Colts front office saying "we are taking Manning, even IF Leaf has more upside."

As for Luck and RG3, there was not nearly as big of a question about taking Luck over RG3 but there was still lots of people who said that RG3, much like Leaf, had more long term "upside" and potential to develop and blossom into a phenom later down the road. However, pretty much everyone agreed that Luck was a "can't miss" pick.

I don't think you understand this luck versus skill concept. Of course it was a conscious decision. They all are. Teams spend countless hours studying prospects, pouring over reams of data, flying hither and yon to see them in the living flesh and then apply what they believe to be superior football acumen to the mix to take the guy they think will be the best available fit for their team. Sometimes they get it right and sometimes they get it wrong.

Pointing to a single decision where a guy got it right, doesn't really tell us much about whether he was lucky or good. I can tell you that there were plenty of Colts fans complaining about Polian's draft history despite the fact that he hit big with Manning just as there are people here in KC who complained about Carl Peterson's ability to draft even though he hit big with Derrick Thomas.

[Consistent1]

Originally Posted by Jakemall

I've always felt that trading down was the smartest thing to do...look at the Pats.

t
They do live on secon and third rounders, but they fail at times too.

Those guys are just over complicating things to me. Sometimes you have to take the shot to win.

[Tombstone RJ]

Originally Posted by patteeu

I don't think you understand this luck versus skill concept. Of course it was a conscious decision. They all are. Teams spend countless hours studying prospects, pouring over reams of data, flying hither and yon to see them in the living flesh and then apply what they believe to be superior football acumen to the mix to take the guy they think will be the best available fit for their team. Sometimes they get it right and sometimes they get it wrong.

Pointing to a single decision where a guy got it right, doesn't really tell us much about whether he was lucky or good. I can tell you that there were plenty of Colts fans complaining about Polian's draft history despite the fact that he hit big with Manning just as there are people here in KC who complained about Carl Peterson's ability to draft even though he hit big with Derrick Thomas.

If you are saying that the way Manning panned out was as much luck as anything else I think you are wrong. When a team spends countless hours on a draft prospect and does indeed draft that guy, but the guy craps the bed and just sucks, are you suggesting this is "luck" too?

Every front office is a little different. What Reid and Co. see as a good draft prospect for their system will not be the same as what Elway and Co. see as a good prospect for their system. Now, is their some luck involved with things like injuries and a whatnot, sure. However, if a team does it's due diligence on a player, that luck factor should be minimized.

Where luck really plays a part IMHO is late round picks that turn into football gods like Tom Brady or Terrell Davis. Yes, indeed, those were some lucky picks as you are kidding yourself if you think Bill B. and Mike Shanahan knew what they were getting when they drafted those guys.

[Mr. Laz]

Originally Posted by Cave Johnson

Study: Trading up in the draft is a sucker's bet

I disagree/Julio Jones

I disagree too/RGIII

[DaneMcCloud]

Originally Posted by patteeu

Pointing to a single decision where a guy got it right, doesn't really tell us much about whether he was lucky or good. I can tell you that there were plenty of Colts fans complaining about Polian's draft history despite the fact that he hit big with Manning just as there are people here in KC who complained about Carl Peterson's ability to draft even though he hit big with Derrick Thomas.

People bag on Peterson but the fact remains that he drafted three Hall of Famers in Derrick Thomas and Will Shields, (it's just a matter of time) in addition to Tony Gonzalez, who is a lock.

Jared Allen has a decent chance and Derrick Johnson has a slim chance.

[DaneMcCloud]

Originally Posted by Jakemall

I've always felt that trading down was the smartest thing to do...look at the Pats.

The Pats have been awful in the draft, even before Pioli left.

Gronkowski fell for a reason (medical) and what's happened with him? Hernandez fell for a reason (character) and what's happened to him?

Belichick has been awful at nearly every position.

[DaneMcCloud]

Originally Posted by Mr. Laz

I disagree/Julio Jones

I disagree too/RGIII

Carl Peterson and Tony Gonzalez disagree, too.

[WhawhaWhat]

Originally Posted by patteeu

It was obviously the right pick, but it's not obvious whether it was lucky or a result of superior drafting skill. Same with the Luck pick, which was made by an entirely different regime than the Manning pick, btw.

I'm fairly certain that Luck was involved with at least one of the Colts picks. Get it?!?

[O.city]

It might have something to do with the fact that "for sure" players aren't traded up for because teams want to draft them? if that makes sense

[patteeu]

Originally Posted by Tombstone RJ

If you are saying that the way Manning panned out was as much luck as anything else I think you are wrong. When a team spends countless hours on a draft prospect and does indeed draft that guy, but the guy craps the bed and just sucks, are you suggesting this is "luck" too?

Every front office is a little different. What Reid and Co. see as a good draft prospect for their system will not be the same as what Elway and Co. see as a good prospect for their system. Now, is their some luck involved with things like injuries and a whatnot, sure. However, if a team does it's due diligence on a player, that luck factor should be minimized.

Where luck really plays a part IMHO is late round picks that turn into football gods like Tom Brady or Terrell Davis. Yes, indeed, those were some lucky picks as you are kidding yourself if you think Bill B. and Mike Shanahan knew what they were getting when they drafted those guys.

I'm not saying that drafting Manning was luck. I'm saying that it's not clear whether it was luck or not and that your confidence that it wasn't isn't well founded.

[Eleazar]

There's not as much distance between the #1 and #10 overall picks as there used to be.

If you look at all the #1 overall picks back to the last one that is still active, Manning...

2012 Andrew Luck
2011 Cam Newton
2010 Sam Bradford
2009 Matthew Stafford
2008 Jake Long
2007 JaMarcus Russell
2006 Mario Williams
2005 Alex Smith
2004 Eli Manning
2003 Carson Palmer
2002 David Carr
2001 Michael Vick
2000 Courtney Brown
1999 Tim Couch
1998 Peyton Manning

For the most part, you have "consensus" #1s at the time they were drafted.

Excepting Andrew Luck (too soon to judge), how many of these guys were worth a #1?

Mario Williams is probably the most recent one who delivered on #1-type potential. Eli Manning did, and Peyton Manning did.

That's really it.

[Tombstone RJ]

Originally Posted by patteeu

I'm not saying that drafting Manning was luck. I'm saying that it's not clear whether it was luck or not and that your confidence that it wasn't isn't well founded.

hmm... well if it was luck then Polian would have flipped a coin, head's it's Manning, tails it's Leaf and the coin would have landed on heads, right?

How much luck will be involved if Fisher pans out or if he craps the bed?

[notorious]

The Leaf/Manning debate was 50/50.


Some people want to rewrite history.