How much is Jay Cutler, the commodity, worth?
 

Brees, Ryan, Rivers, Cutler and Manning (and maybe Brady, too)

Posted by Chase Stuart on Monday, March 23, 2009

The biggest story in the NFL these days not involving the NFL draft has to do with the bizarre circus involving Jay Cutler. Many have written about everything from Cutler’s production and his psyche to macro thoughts on Bill Belichick disciples; I have nothing to add there. I’d rather take on an impossible task and take a statistical look at how Jay Cutler ranks among other QBs.

And when I say how Jay Cutler ranks, I mean how Jay Cutler 2009, 2010, 2011, 2012, and so on, ranks. And that’s why it’s an impossible task. I can’t predict what will happen in three years. You can’t predict what will happen in four years. We won’t know how good Cutler is over the next five years until five years from now. It’s all a guessing game, but that doesn’t mean we can’t refine our guessing. There are obvious and not so obvious flaws in the approach I’m about to outline, and I’ll do my best to explain them.

What I’m trying to figure out is how valuable is Jay Cutler, the commodty, in March 2009? For example, we know we’d rather have Cutler than Tarvaris Jackson and we’d rather have a 26 year old Peyton Manning than a 26 year old Jay Cutler. But to determine his value — his trade value, if you’re a Broncos fan — you need to know what he’ll do in the future. And as Yogi Berra once said, it’s tough to make predictions, especially about the future. Consider:

• At the end of the 1970 season, Billy Kilmer was 31 years old and had just 11 career wins to his name. In 1970, he threw 6 TDs and 17 INTs. Who would have guessed he would have been one of the best QBs in the NFL over the next half-decade?

• At the end of the 1995 season, Rich Gannon was 30 years old and had started just four games over the past three years. Who would have guessed that he would make four Pro Bowls over the next seven seasons?

• Stop me if you’ve heard this one. QB is drafted 330th overall. He throws 1 TD and 7 INT his rookie season. He throws 1 TD and 3 INT and has an 0-2 record his sophomore season. You probably wouldn’t think he’d still be in the league five years after that, but in his seventh season Brian Sipe was named a first team All Pro.

• Kurt Warner. Tom Brady.

This goes the other way, too, of course.

• Archie Manning struggled early in his career, but the #1 pick was very good at ages 29, 30 and 31. Who would project him to throw 15 TD and 30 INT the rest of his career?

• Daunte Culpepper, at age 27, had the most total yards in NFL history. Then he became Daunte Culpepper. And yes, I know he lost Randy Moss and battled injuries.

• Mark Rypien won a Super Bowl and had one of the better seasons in passing history in 1991. He was just 29 years old. He never came close to duplicating that success.

• Greg Landry was a first round pick who had big years as a runner and passer in 1971 and 1972. He was 26 years old in March 1973, and he may have been the single best QB prospect in the NFL at that time. When you consider his rushing, he’d been more productive than Manning or Bradshaw or Griese, he was younger than Tarkenton and Namath, and he had a better pedigree than Ken Anderson. Landry stuck around for awhile but there were about 15 QBs who outperformed him from that moment on.

• Steve Bartkowski was a former number 1 pick who made the Pro Bowl in ‘80 and ‘81 and had an incredible 22 TD/5 INT ratio in 1983. He was 31 years old, but he threw for fewer than 4,000 yards the rest of his career.

None of this is breaking news; we all know it’s difficult to predict one year down the road, let alone five or ten. But it’s important to set the stage before we answer the key question: if you could have any QB in the NFL right now, for your favorite team, who would you pick? Can statistics and past history guide us?

There are a ton of factors that you would want to use to predict future QB success, but there are three that seem most prominent: age, past production and draft value. Unfortunately, each of these are complicated variables and require a full description, but first, let me describe how we measure future QB success. If you hate reading the details, skip to the end for the QB list. [author's methods in next post]

Here are the top 25 projected QBs going forward in the NFL as of March 2009, but using only QBs who had 200+ attempts in 2008. The age is how old the player was during the 2008 season.

08Val Age AgeV DraftV 3YrVal Proj Rk Name
2157 29 537 11.8 1861 5049 1 Drew Brees
1245 23 552 59.4 1245 5009 2 Matt Ryan
1919 27 581 33.5 1429 4829 3 Philip Rivers
1495 25 591 34.4 1169 4277 4 Jay Cutler
1501 32 430 7.3 1702 4096 5 Peyton Manning
1217 28 563 1.5 1372 3699 6 Tony Romo
875 27 581 43.9 648 3270 7 Eli Manning
484 23 552 73.2 288 3127 8 JaMarcus Russell
492 26 591 30.1 727 3106 9 Ben Roethlisberger
1422 25 591 26.6 733 3016 10 Aaron Rodgers
1234 32 430 6.4 1205 2889 11 Donovan McNabb
477 23 552 36.8 477 2538 12 Joe Flacco
731 27 581 19.6 614 2495 13 Jason Campbell
1030 27 581 9.9 728 2489 14 Matt Schaub
1721 37 254 0.3 1305 2207 15 Kurt Warner
645 30 505 4.3 801 2179 16 David Garrard
1477 32 430 3.7 925 2143 17 Chad Pennington
663 25 591 13.0 490 2050 18 Trent Edwards
1013 26 591 3.4 495 1789 19 Matt Cassel
74 25 591 4.6 429 1665 20 Derek Anderson
539 26 591 10.2 302 1523 21 Kyle Orton
1072 33 391 0.3 752 1468 22 Jake Delhomme
553 24 578 4.9 354 1444 23 Tyler Thigpen
528 28 563 1.5 410 1413 24 Shaun Hill
462 28 563 7.0 277 1254 25 Seneca Wallace

How does that list look to you? This is my best attempt to, using just objective data, figure out which QBs would have the most value either on the open market or through a trade. All three input variables were highly significant, which means they are all certainly correlated to future production. The R^2 was just 0.35, which isn’t very high, but I’m not sure if you can come up with a formula to make it any higher. There’s a ton of randomness in future production, and if 35% of it can be predicted through this formula, that’s pretty good. Before we conclude, let me throw some general thoughts out there on this list:

1) If we ignore Rodgers’ 2006 and 2007, and just use last year for his 3YrVal, that would give him a projected score of 4654, and he’d move into 3rd place on the list. Very interesting. Doing the same analysis with Cassel bumps him just north of 3000, where Rodgers currently is.

2) Joe Flacco’s ahead of Kurt Warner. That makes some sense to me. We’re basically projecting one or two big years out of Warner versus eight good years out of Flacco.

3) Matt Ryan (along with Rodgers if we tweak his score) comes in too high, I think. Why? My buddy Maurile would recommend using some sort of Bayes’ theorem analysis, because being great over a small number of attempts is not as convincing as being great over a large number of attempts. As I’ve currently structured it, three great years will have the same score as one great year, if you only play one year. Usually, that’s not a problem, but when you have the greatest rookie season ever (or you edit Rodgers’ career), things get dicey. Suffice it to say, while I love Ryan, I’m not convinced just yet that he’s mega elite. I love Ryan as much as anyone, but putting him at #2 scares me a little bit.

4) Look over there at JaMarcus Russell. He’s up there over Big Ben! Two of the three factors point Ben’s way — being older actually helps him, since he is entering his prime, and obviously he’s already better. The only thing pointing Russell’s way is that #1 draft pick status. Is that right or wrong? Here’s how every QB drafted #1 overall since the merger (along with Steve Young, a supplemental #1) ranked among QBs in their first two seasons.

RookYr Yr1 Yr2
JaMarcus Russell 2007 82 24
Alex Smith 2005 81 26
Eli Manning 2004 79 10
Carson Palmer 2003 -- 24
David Carr 2002 84 25
Michael Vick 2001 29 5
Tim Couch 1999 35 27
Peyton Manning 1998 20 3
Drew Bledsoe 1993 25 12
Jeff George 1990 36 59
Troy Aikman 1989 81 63
Vinny Testaverde 1987 49 81
John Elway 1983 74 16
Steve Bartkowski 1975 23 70
Jim Plunkett 1971 12 68
Terry Bradshaw 1970 65 21
Steve Young 1984 -- 67

Sure, Russell looks bad. But so did Troy Aikman. And Vinny Testaverde. And Terry Bradshaw. So did Steve Young. John Elway wasn’t much better. I have to say, it’s very counterintuitive to me to suggest that Russell is going to be better than Roethlisberger, going forward. On the other hand, you could probably have made the same argument about Jim Everett and Troy Aikman in March 1991, too. As for Ben and Russell, I think it’s at least arguable that Ben’s statistics understate how good he really is. If his 3YrVal was a bit higher, he’d easily be ahead of Russell and Manning.

5) Notice anyone missing? I don’t really know what to do with Brady, but his score after 2007 was 5179. Giving him an extra year of age would drop him to 5008, but we can’t ignore that he suffered a serious injury. Who knows how he’ll come back, and I don’t really know where’s a good place to put him. If he’s healthy, though, he’s right there up with Brees. Carson Palmer was right behind Brady in 2007, so if he’s healthy, he’s got to be in the mix, too. If we age him a year based on his ‘07 numbers, he’d project at 4471.

Continued:


Let me close with a subjective list, using the numbers above as my guide (along with the changes I mentioned above). I won’t try to be so exact, but I’m going to group the top 25 or so QBs in the NFL into tiers.
Build your team around them (6): Drew Brees, Tom Brady, Peyton Manning, Philip Rivers, Jay Cutler and Matt Ryan. These six QBs are at their own level, IMO, as no one matches past production and youth the way they do. You build your team around any of these guys, and all would be a good bet to be the NFL MVP in 2009, 2010 or 2011.

You’re in love (5): Aaron Rodgers, Tony Romo, Eli Manning, Ben Roethlisberger, Carson Palmer. It wouldn’t take you more than a second to find fault with any of these guys. That said, you’ve got to feel like you can go to the playoffs anytime in the next five years with this guy at the helm.

I really, really like you (3): JaMarcus Russell, Donovan McNabb, Joe Flacco. These guys are a half step below the guys above and a half step above the guys below. You’re happy with your QB outlook, but you’re not going to puff your chest too much. While Russell hasn’t received nearly the praise that Flacco has, Russell put up at least equal numbers last season despite playing for a worse team and despite being eight months younger than Flacco. Warner arguably deserves to be in this group, but I don’t think he has enough left in the tank.

You like what you’ve got (7): Jason Campbell, Matt Schaub, Kurt Warner, David Garrard, Chad Pennington, Trent Edwards, Matt Cassel. These guys don’t have a lot in common. Warner may be an All Pro next year, but you’re going to be looking for a new QB very soon. Garrard and Pennington have had big years while Schaub and Campbell have shown flashes. Edwards and Cassel are inexperienced but have done okay so far. The whole group is a mixed bag, and if this guy is your QB, you can’t even guarantee that he’ll be in the league in five years. I suppose a healthy Matt Hasselbeck could join this tier, too. Based on just 2008, Warner and Pennington are the obvious class of the group, but both have long histories.

Can I really trade for Jay Cutler? (6): Derek Anderson, Kyle Orton, Jake Delhomme, Tyler Thigpen, Shaun Hill, Seneca Wallace. The less said about these guys, the better.

So what do you think? How would you rank the QBs?

so it's down to Continental Divide vs. Blvd..... amount of cases to be determined.

haha. Jamarcus Russell will never be worth a squirt of piss in the NFL

Methods:

First I calculated each QB’s adjusted net yards per attempt (passing yards + 20*TDs - 45*INT - sack yards lost) / (passes + sacks) metric. Then I compared that ratio to “replacement level”, defined as 75% of the league average. Then I multiplied the difference between the QB’s ANY/A and replacement level by the QB’s number of pass attempts plus sacks to get a measure of “adjusted yards over replacement.” Then, I added to that number every adjusted rushing yard (rushing yards + 20*rushTDs) over four yards per carry. So if a QB had 100 carries, 500 yards and 5 TDs, that would be plus 200 adjusted yards. Whether or not this formula is perfect isn’t that important — for what we’re looking for, with hundreds of QBs, something that’s generally correct is all we need. We don’t need to know specifically if Troy Aikman was better than Jeff Garcia, but just that both were better than Danny Kanell and Bobby Hoying.
Measuring QB success or value in a given year isn’t enough, though. We want to know how they’ll do for awhile, although we also want immediate success. To grade QB value for the long term, I took 100% of their production in Year N+1 (2009, for Jay Cutler’s purposes), 95% of their value in N+2, 90% in N+3, and so on, for 8 seasons. This will thus reward great immediate production and sustained levels of strong play. This weighted measure of eight years of production will be the output variable in our regression formula. Not surprisingly, Manning in March 2000 (the year he turned 24) and Marino in March 1984 (the year he turned 23) come out as the top two scores.
Age:: The effects of age are obviously nonlinear; going from age 23 to 24 is good; going from 35 to 36 is bad. I looked at the top 50 QBs (based on NFL production, using something similar to the metric I used in the Greatest QB Ever series) drafted since 1970, that are no longer active, to determine the general “dropoff” rate for a QB. Here are the results:

21 12.4
22 34.6
23 53.1
24 68.1
25 79.9
26 88.8
27 94.9
28 98.5
29 100.0
30 99.5
31 97.3
32 93.7
33 88.9
34 83.2
35 76.8
36 69.9
37 62.9
38 56.0
39 49.4
40 43.4
41 38.3
42 34.2
</pre> This shouldn’t be too controversial; it says that QBs peak from ages 28-31, and a QB at 36 is about as good as a QB at age 24. However, we can’t just use these numbers as the inputs in the regression formula because we’re trying to predict a weighted value of eight years worth of scores. So the value of being 24 corresponds to 100% of the value of being 25, plus 95% of the value of being 26, and so on. Here are those numbers:

21 12.4 452
22 34.6 510
23 53.1 552
24 68.1 578
25 79.9 591
26 88.8 591
27 94.9 581
28 98.5 563
29 100.0 537
30 99.5 505
31 97.3 469
32 93.7 430
33 88.9 391
34 83.2 352
35 76.8 315
36 69.9 282
37 62.9 254
38 56.0 225
39 49.4 196
40 43.4 172
41 38.3 151
42 34.2 132
</pre> Therefore, the ideal range of ages where your average QB has his best days ahead of him is somewhere between 24 and 27 years. And that makes a lot of sense. And whereas the table above said being 36 was slightly better than being 24, now being 24 is much better than being 36. And that makes sense, too.

Finally: Methods Continued:

Past Production: I used the same formula to measuring past production as I did for measuring future production; adjusted net yards added over replacement value. However, I didn’t want to limit myself to 2008, so I used a weighted average of the last three years. So for Philip Rivers, he gets 3*2008_value plus 2*2007_value plus 2006_value, all divided by six. For JaMarcus Russell, he gets 2*2008_value plus 2007_value, divided by three. For Matt Ryan, he simply gets his 2008 value. This becomes a little unfair for someone like Aaron Rodgers, who is treated like a star in 2008 and a nothing in ‘07 and ‘06. That’s not an accurate portrayal of what happened, but we can tweak that later.
Draft Value: I used the numbers here to assign a value to each slot in the draft. I feel really comfortable with those values, but there’s one big problem: a player’s draft value is really important when they’re young but not very important when they’re old. I made some rough, back of the envelope calculations as to a good multiplier for draft value at each age:

22 1.0
23 1.0
24 0.9
25 0.8
26 0.7
27 0.6
28 0.5
29 0.4
30 0.3
31 0.2
32 0.1
33 0.1
34 0.1
35 0.1
36 0.1
37 0.1
38 0.1
39 0.1
40 0.1
</pre> This just quantifies what my gut says, although I have no idea how accurate it really is. Deriving a good formula for this could be a whole separate blog post, so I’m going to put that off for another day. So I simply multiplied each player’s draft value (as derived in that post) times their multiplier based on age. There’s a problem with this, though: it makes the “draft value” variable for a 31 year old Brett Favre equal to the draft value variable for a 24 year old Tom Brady. That’s beacuse Favre’s an old guy that used to be a high pick and Brady’s a young guy who is a low pick. That’s what I ended up doing, but I don’t love it. Unfortunately, I’m not sure if there’s a better way — using regression analysis — to handle this.
If you’ve made it this far, the payoff is about to come. We now have our input variables — an age variable, a draft value variable that decreases over time, and a past production variable. The output is a weighted average of performance over the next eight seasons. What’s our dataset? Every QB who entered the league since the merger, but excluding all player-seasons since 2003 and any QB born after 1977. Further, I only looked at player-seasons with at least 200 pass attempts.
What’s the formula that best fits that curve? It won’t mean much to you, but here it is:

-1972 + 4.21*age + 28.6*draft + 2.38*pastproduction

I agree.

Is there a cliffs notes version?

he's back for another off-season update

<object width="480" height="295"><param name="movie" value="http://www.youtube.com/v/uQ8EbKwbmVw&hl=en&fs=1&rel=0"></param><param name="allowFullScreen" value="true"></param><param name="allowscriptaccess" value="always"></param><embed src="http://www.youtube.com/v/uQ8EbKwbmVw&hl=en&fs=1&rel=0" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="480" height="295"></embed></object>

reeruned kids love to post youtube videos nonstop. Its a fact.

That is hysterical. I could use a laugh like that on a daily basis. Bookmarking this link for future reference!

A guy with a career losing record is the 4th most valuable QB in the league.
ROFL

I'm working my way through this...

"The R^2 was just 0.35, which isn’t very high, but I’m not sure if you can come up with a formula to make it any higher. "

That's terrible.

"Whether or not this formula is perfect isn’t that important — for what we’re looking for, with hundreds of QBs, something that’s generally correct is all we need. We don’t need to know specifically if Troy Aikman was better than Jeff Garcia, but just that both were better than Danny Kanell and Bobby Hoying."

Guy sets the bar pretty high, huh?
If he wanted it to make sense, why is he simply devaluing each QB's play next year by 5% continually? He should run a regression and find out exactly what impact every age has on a QB's change in efficiency.--I didn't read far enough, it sounds like he did this? not clear
I haven't gotten to the methods yet, but I think this thing is flawed from the start. I think you HAVE to find a way to account for the offense a guy has around him. see Philip Rivers

That whole part about draft value is just dumb. It doesn't make sense at all and he just made it up and stuck it in there. At least he admits it, though. Shockingly, according to their final equation they found draft value to have the MOST impact. I think that's just because the rest of it is so flawed. Since their formulas don't line up, obviously NFL teams will do a decent job of guessing how players will do.

I tried, but I can't fight through that one...

thanks i didn't have an hour to read it.

not that i would have after i saw Russell ranked above Roethlisberger