What are NFL draft picks really worth (and do they help teams win)?

One of the biggest moments in the history of the NFL Draft took place almost as far from it as possible. On October 12, 1989 – 171 days after Everett Ross became Mr. Irrelevant as the last pick of the 1989 Draft and 192 days before Jeff George kicked off the 1990 edition – the Dallas Cowboys sent Herschel Walker and a handful of picks (two 3^rd rounders, a 5^th and a 10^th) to the Minnesota Vikings in exchange for five players, three 1^st round picks, three 2^nd round picks, a 3^rd round pick and a 6^th round pick. Dallas’ multiple 1990s Super Bowl runs were powered by players such as Emmitt Smith, Russell Maryland, Alvin Harper and Darren Woodson who either came directly from the picks or as a result of further trades involving those picks.

Mike McCoy, who owned approximately 5% of the team, had been a business partner of majority owner Jerry Jones in the oil business. The team was looking for a way to systematically value their cache of draft picks, and McCoy was the one to do it. According to a 2004 Dallas Morning News article, McCoy spent two days graphing the actual trades that had taken place over the past four years. He found that the trades appeared to fit a trendline overlaid on the graph. This trendline became the basis for the Draft Value Chart[1].

The Chart provides a value for each pick in the form of unitless “points” assigned decreasing from 3000 for the number one overall to, depending on which source you consult, 0.4 points for the 256^th pick or 2 points for the 224^th pick[2]. The decrease in point values is extremely steep at the top of the draft with the value dropping by 50% to the 7^th pick and by another 50% to the 24^th pick, leaving it only 25% as valuable as the number one selection for trade purposes.

Since the early 1990s the Draft Value Chart has made its way through the NFL and become the basis for draft pick value on nearly every team. The assistant coaches and assorted front office employees from the Cowboys took it around the league when they left the team. Research conducted by Cade Massey and Richard Thaler plotting actual trades found that prices aligned closely to The Chart with the deviation from the chart dropping significantly and trade volume increasing in the years after it became well known[3]. In other words, once the chart became widely accepted teams did not vary from the assigned values.

The Chart over time

A funny thing happened, however, just as The Chart was taking its place at the front of draft war rooms around the league. The salaries awarded to NFL top draft picks skyrocketed, and those at the very top of the first round were the biggest beneficiaries of the change. Guaranteed money promised to the first pick in the draft rose from Troy Aikman signing with the Cowboys in 1989 for 6 years and $11.2 million with a $2 million bonus through Sam Bradford’s $78 million total, $50 million guaranteed, 6 year monster contract with St. Louis in 2010.

Agents and teams found a straightforward negotiating strategy: take last year’s contract and add a bit. For a baseline against which to compare the rise in rookie salaries, the NFL salary cap grew from $35 million to $123 million between when it was introduced in 1994 and 2009 (2010 was uncapped). This represents a growth rate over that period of roughly 9% while the guaranteed money to the first pick grew from Dan Wilkinson’s $5 million to Matthew Stafford’s $42 million at a CAGR of 15.2%[4]. If teams wished to dedicate more resources to top rookies in the post-1994 environment then they would automatically have to decrease the amount of resources dedicated to veterans to ensure that they stayed under the salary cap.

Figure 1 – Guaranteed money (dotted line) and salary cap (solid line) 1994=100

This difference in growth rates became one of the main drivers of addressing the situation. Absent a change in the overall salary cap a reduction in the amount of money going to rookies results in more money going to veterans. A measure to redistribute money from non-members to current union members was relatively palatable given the alternatives in the 2011 CBA negotiations. The salary floor in the 2011 CBA, along with several other provisions, offers assurance for the players that owners won’t pocket the savings.

As top picks became more and more costly, teams were stuck between the necessity of signing the pick and the reality of the salary cap constraint that the pick represented. A brief examination of the bargaining power on each side reveals significant leverage for the player.  While the player would lose out on one year of salary if no agreement were reached, the team would suffer a major public relations problem likely to cost even more than the player’s salary (e.g., irate fans, loss of reputation), the additional cost of continued uncertainty because the contract situation would hang over the team (e.g., management time) and, finally, a lack of suitable replacement options for that player’s expected production because free agency is largely complete by the time of the draft.

Some teams responded to this state of affairs by attempting to pre-negotiate deals with top players, buying off the possibility of a holdout in exchange for picking a player first. While this was only available for the top pick in the draft, it did result in a notable decrease in the rate of salary increase for those players, but in general the salaries kept on going. Figure 2 shows year over year change in guaranteed compensation to the top pick, with those who agreed to a deal prior to the draft shown in a lighter shading.

Figure 2 – Year over year change in first pick guarantee (Dark – No pre-draft contract, Light – Pre-draft contract)

Not only are the lighter bars much lower than the darker ones (10% vs 23%) but that the lighter bars are only as high as they are because of the increases won by Carson Palmer (+29%) and Matthew Stafford (+39%) in the years following another pre-draft contract agreement – a sort of snap-back in response to the suppressed increase from the prior year. This anecdotal evidence suggests that negotiating a deal early was helpful, but not a cure-all for the inflation in guaranteed money. Through the uncapped year of 2010 (Sam Bradford, +20%) salaries continued their upward trajectory.

What happened with respect to the Draft Value Chart was that teams simply stopped trading top picks. Because the distortions in rookie contracts were concentrated in the very top of the first round, those are the picks that varied the most from their Chart value. From 1991 to 2000, there were 12 trades involving a top five pick as the best pick. From 2001 to 2005, there were only three of these and from 2006 to 2010 there was only one – that one involved the number five pick. The overall number of first round picks dealt actually increased from the 1991 to 2000 period to the 2001 to 2010 period from 67 to 80 at the same time as the decline in trades of top picks[5]. This provides some powerful anecdotal evidence that top picks had exceeded a threshold where teams were unwilling to go. Only special circumstances such as Eli Manning, the consensus number one pick who refused to play for the team in the top overall slot, caused trades to go through. As many in the media theorized that the best option would be to trade down for the massive value promised by The Chart, teams found that there were no willing buyers.

2011 CBA

One of the biggest changes of the 2011 lockout and subsequent collective bargaining agreement was to set an individual salary cap for draft picks much closer to the way the NBA does (and in a way that MLB Commissioner Bud Selig only dreams about). The value allocated to each pick is set by the CBA – allocated on a team-by-team basis based on which picks they ended up with in that year’s draft – and teams’ flexibility to give additional money in signing bonuses, roster bonuses and various other methods by which they increased contracts in the past was cut off. Picks are given a disincentive to hold out because there is nothing more to be gained with the maximum salary and number of years known before the draft. The only drama in the summer of 2011, post-lockout, was the number of guaranteed years in the bottom half of the first round. Teams had believed that the top 16 picks would get four years guaranteed while the remaining first rounders would get only three. After Tampa Bay gave Adrian Clayborn a full four year guarantee at pick 20, the entire bottom half of the first round and the 17-19 picks in particular were locked in a staring contest with their respective front offices. The outcome was that 17-19 and 21 took 3.5 years guaranteed while 22+ ended up with 3 year guarantees[6].

Figure 3 – Value per pick changes from 2010 to 2011

Looking at figure 3, we can see the change from 2010 to the first year of the new CBA in 2011, the pattern of salary increases throughout the period is finally reversed. Not only did every single pick become more valuable, the top ten gained an average of 83% relative to the 2010 draft while 11-20 and 21-32 followed with 37% and 30%, respectively. The 2011 CBA reflected a concerted effort on the part of the teams of the NFL to recenter the value of individual draft picks in a way that made them more attuned to the NFL’s “competitive balance” mindset. The change in value from 2009 to 2011 is a still-impressive 52% improvement for the top ten (85% for the number one pick) with improvements of 20% and 14% for 11-20 and 21-32, respectively.

It seems relevant to look at the salary cap vs guaranteed money chart that demonstrated the extreme divergence from 1994 to 2009 with the inclusion of new data from 2011. When the final data point with Cam Newton’s $22 million and the new salary cap number of $120 million is included in the chart as seen in figure 4, there is significant reversion back towards the relationship that existed as of 1994, back into the territory where teams were willing to deal. In terms of relative difference between the cap and first pick guaranteed money, 2011 ends up on a level with 2003, 2000 and 1995 in terms of their difference from the 1994 starting point. It remains to be seen whether the restraints into which the owners placed themselves will hold or if salaries will find a way to creep up faster than the cap as they did throughout the prior 15 years.

Figure 4 – Guaranteed money (dotted line) and salary cap (solid line) 1994=100

The point here is not to pile on The Chart, it should actually be more relevant now as the rookie salary situation is close to what it was in the 1990s compared to the overall salary cap. As mentioned previously, it was developed by looking at trades and not the actual value provided by a pick and so it simply institutionalized any errors of valuation taking place during the late 1980s and early 1990s. Teams that take a proactive approach to relative draft values and capitalize on the relative changes in value can gain a competitive advantage, potentially a very large one, in a very competitive league.

Revisiting Player Value

The new CBA represents a lifting of the unofficial moratorium on trading top picks as demonstrated by the three trades involving top five picks in the 2012 draft[7]. No longer do GMs have to worry about a 20%, 30% or 40% increase over the prior year’s guaranteed money for an early first rounder. They do not have to imagine trading for a player who proceeds to hold out for record-breaking contract and putting a season on hold while fans become more and more impatient. For the draft and The Chart, this means a chance to redefine the valuation approach and adjust it to the way players are valued in the modern NFL.

First of all, any new Chart needs to take into account the information contained in the salary cap. The salary cap forces teams to make constrained maximization decisions by tying additional resources to a given position with a reduction in resources available to other positions. In attempting to define a chart for draft values, this new version of The Chart is aiming at the value of the picks relative to each other. This will mean that the initiating team in a trade may have to provide additional value to persuade the other team to do the trade rather than simply making the selection.

If a pick is openly auctioned or a bidding war emerges, the “winner” is likely to overbid because of the Winner’s Curse which leads to overpayment relative to expected value in competitive bidding situations[8]. The Chart as constructed by Mike McCoy was already built to account for these biases because it included past trade data only, not production. The negative of doing this was that once teams were anchored to The Chart as a baseline, the overpayment creeps back in during bidding wars as teams know the prescribed value is likely to be too low to win the pick.

Contract Lifecycles

Each of the three major American sports leagues (with apologies to the Raptors and Blue Jays, and hockey) has a particular way of dealing with rookies. It is worth taking a look at these methods to get an idea of how to quantify what a draft pick really brings to a football team.

Baseball allocates the first six years of a player’s Major League career to the team. Arbitration means that the player has some leverage to improve their situation – particularly in later years – but they still receive a salary below their market rate during this period. Once they have completed their first six seasons a player is a free agent in the truest sense. Any team can offer him a contract for any amount or length of seasons[9].

In the NBA, draft picks receive a contract specific to their slot with no negotiation. These contracts run for two years with two additional years of team options. For exceptional talents like Dwyane Wade or Chris Bosh, the options are exercised and the drafting team receives four years of production at a below-market rate. Once the player reaches free agency there are more controls than in baseball, although it would be difficult to have fewer. The team holding the player’s rights can, in most situations, offer them an additional year on their contract and slightly higher increases in all of the contract years, although a player is still free to make a decision to leave. This alone does not help the team capture surplus value, but the salaries for individual players are capped depending on tenure in the league so the salary for a player of Wade or Bosh’s caliber will remain below their actual value in terms of production. Additionally there are a few situations in which a player can be granted restricted free agency, at which point their team will have a right to match any offer they sign with another team[10].

The NFL rookie system has already been covered, but the free agency system helps inform the value included in draft picks. Restricted free agency is a feature of the NFL for players with three or fewer seasons of accrued service and whose contracts have expired, with teams tendering a contract at a value that corresponds to a round in the draft. If another team attempts to sign the player, the first team will have the right to match. If they choose not to match then they receive the signing team’s draft choice in the round of the tender. With draft pick compensation much more manageable in the new CBA, it has been very rare for teams to sign another team’s restricted free agent[11]. The other, bigger, weapon for NFL teams to use is the franchise tag and the similar but more rarely-used transition tag. Both of these tags deny the player the right to sign with another team. In compensation for this the salary for the one-year offer is relatively high (average of the top five salaries at a player’s position for the franchise and average of the top ten for the transition)[12].

Because it limits the player’s rights, the franchise tag is extremely unpopular when used. This unpopularity – along with players’ desire to avoid being “franchised” – grants significant bargaining power to the teams. A team can effectively prevent a talented player from leaving the team indefinitely. In practice it typically leads to a long-term deal with the player in question. By agreeing to these deals in advance, a team can even reserve the once-per-season franchise tag for leverage with other players. Given this structure it makes more sense to look at the career value of a player rather than the value only in the first few seasons because teams effectively have an option on the entire career of elite players.

The Value of a Player

The game of football involves 22 starters who each are on the field at most 50% of the time, has multiple positions that accumulate almost no traditional statistics as well as others whose elite players may demonstrate their value by changing the behavior of other teams and not gaining stats themselves (e.g., not throwing toward a top cornerback, putting eight men in the box against a top running back) and has far fewer “observations” than other sports because teams only play 16 games in the regular season, so it takes a bit of work to get a data set that conveys the value of a player. In lieu of doing this work, we can lean on the combined product of thousands of professionals working for NFL franchises and calculate relative value based on actual salaries in the league[13].

One of the issues with doing this analysis in 2012 is that the contract data are not fully comparable due to an uncapped year in 2010 and the major changes instituted in the 2011 CBA. To move back to the last period of stability in contracting and team salary construction we will use the salaries from the 2008 and 2009 seasons as the data set for assessing value.

The next question with the salary data is whether to include all players or only those who have signed a contract after their rookie contract expires (excluding undrafted free agents). Because the contracts that go to the rookies can still be assumed to vary depending on position (i.e., teams do not typically take a center very early in the draft because they will have to pay a price based on the draft pick that exceeds where centers can be had in the free agent market) and expected production (i.e., a receiver taken with the third overall pick is expected to perform at a high level) the full set of salaries would seem to be more representative. An additional deciding factor is that the NFL is a league of relatively short careers. Looking only at those players who are 25 or 26 and up would significantly distort the salaries because it only looks at those players who are still in the league at that point. Understanding that the results will be imperfect either way, looking at the entire population of players better represents the money allocation decisions that teams were making during the sample period.

Since we are using annual statistics to try to capture player performance, we have to simplify in order to get a meaningful result. To begin with, the players drafted before the beginning of the salary cap in 1994 must be excluded. Teams at that time could afford to devote more resources to one position without automatically being forced to devote fewer resources to another. In terms of position-specific performance, the metrics that capture elite performance are election to the All-Pro team and the Pro Bowl. While looking at the Pro Bowl involves less contention – each position outside of fullback has at least four players selected – the All-Pro team provides a useful additional layer of granularity for those players at the top. NFL teams, it turns out, are reasonably good at identifying elite players at the top of the draft as demonstrated in figure 5.

Figure 5 – Pro Bowl seasons per career, by draft pick (1994-2011)[14]

Pro Bowl seasons per career appear to vary strongly by draft pick (punters and kickers are excluded from these figures). The level at which a player was drafted explains 72% of the variation in Pro Bowl seasons in a player’s career. The charts for All-Pro seasons, seasons as a starter and seasons in the league all show similar relationships between draft position and performance.

The number of opportunities to play represents an omitted variable that, if known, would likely help refine the model. While coaches frequently talk about positions being open for competition, there is a great deal of pressure to play top picks and so those players have the most opportunity to establish themselves. Unfortunately, we don’t have any insight into the counterfactual that would happen if 7^th rounders had the same opportunity to play as 1^st rounders, so we will proceed with what we have.

To link this elite performance to value, we need to layer on the value placed on each position. Different positions are compensated very differently in the league as is demonstrated in table 1.

Table 1 – Value per position, by level of performance (2008-2009 average)

Table 1 shows salaries by position for the 2008 and 2009 league years, allocating the highest at each position to the “All Pro” level of value, the next several to the “Pro Bowl” level, then on to the “Starters” and “Others,” respectively. Rather than look at the actual members of the All-Pro or Pro Bowl teams, this considers the top salary or 2 (up to 4 for LBs and DBs) to be the value assigned to a player of that caliber. This assumption accounts for the fact that contracts are signed in anticipation of an uncertain level of services while postseason awards are given out retrospectively based on definite performance. It is as if we could reorder the players and their contracts at the conclusion of the season to match them based on how much they actually contributed. This method still underestimates the actual number somewhat at the top because teams price in a certain amount of uncertainty due to injury or performance: no team is not likely to sign a contract that would only break even in value if a player were an All-Pro in each year.

Several observations stand out from table 1. First, quarterbacks are the clear leaders at all levels in terms of salary cap allocation. The NFL is a quarterback league and the rule changes of the past several years favoring the passing game specifically and offense over defense generally make it difficult to see this changing anytime soon. Defensive end is the clear second place, holding that distinction at all levels. This comes from the same set of changes as getting to the quarterback offers a chance to prevent the most valuable player (in salary allocation) from doing his job. Following on are the other pieces of the passing game on both sides of the ball, with offensive tackles, wide receivers and defensive backs up next in allocation. Members of the running game – defensive tackles, linebackers, running backs and interior linemen bring up the rear ahead of tight ends and fullbacks. Looking to the future, it is probable that tight ends may be in for an increase as several top teams in 2011 featured them more in the passing game than was previously the norm[15].

Draft Value Chart 2.0

Armed with the performance of players and the value of different levels of performance, both stratified into All-Pro,  Pro Bowler, Starter and Other, we can now combine the two into a comprehensive model of the value expected of each draft position.

The first step is to assign a value to the career of each player selected from 1994 to 2011. This is accomplished by multiplying each All-Pro season by the value of an All-Pro season appropriate for that position in terms of % of the salary cap. By repeating this process for Pro Bowl seasons (less any All-Pro seasons), seasons as a starter (less any All-Pro/Pro Bowl seasons) and seasons not starting, we have a complete picture of the player in terms of career value[16].

To demonstrate the process let’s walk through what this looks like for Peyton Manning. Peyton has five first-team All-Pro seasons, each of which is multiplied by the 16.3% salary cap allocation due to the top QB. His eleven Pro Bowl seasons (less the five All-Pro seasons) net six times the 11.4% allocation due to the second through sixth QB. To this we add two seasons as a starter with a 5.5% allocation and one season not starting with a 0.8% allocation. Combining all those seasons gives Peyton 162% cumulative value as a % of the salary cap over the course of his career through 2011.

This process is repeated for all 4,519 picks from 1994 to 2011. While more recent picks are still active, this will affect all draft slots so they are included in the data set[17]. Once this data is compiled, averaged by pick and plotted on a graph it looks something like figure 6.

Figure 6 – Average value per pick (career – solid line, season – dotted line)

With value per career on the left axis and value per season on the right axis, this graph will become the basis for our new value chart just it can be distilled down to a trendline rather than the averages of an 18 year set.

Note that the value per career of later picks (particularly above 100) drifts below the value per season while for higher picks they overlap extremely well. This reflects the longer average careers for top picks. A first round pick averages 6.67 seasons, the second 5.87, followed by 4.95, 4.37, 3.71, 3.10 and finally 2.60 for the seventh[18]. These statistics reflect the higher value of durable players in the draft and the higher skill level of top picks, who can stay in the league longer as they age.

Because the averages represent only 18 drafts we need to smooth the results to get a projected value for each pick. Working from the career value, applicable for both the contracting specifics of the NFL and additional information about longevity contained as noted above, the first portion of the graph is best fit by a logarithmic regression while the latter stages appear to be random around a relatively linear trajectory. Combining these by graphing the first 150 with a logarithmic regression and the last 150 with a linear regression (resulting in an overlap of 45 picks), we take the maximum value of the two lines as demonstrated in figure 7. While not mechanically perfect, this method allows us to account for the different underlying trends in different parts of the draft.

Figure 7 – Average career value per pick with regressions

The r-squared of the career value for picks 1 through 150 is an extremely strong 0.867. For the last 150 picks (numbers 105 to 255) the r-squared is a much-weaker 0.246, indicative of the high volatility of this portion of the draft. A few positions are significant outliers such as the 199^th pick, where Tom Brady’s 102% career value helps pull the average up to 8.9% from a projected career value for that pick of 3.0%.

The next step in the process of turning this curve into The Chart is to remove the value of a replacement player. This is where we need to avoid reaching the conclusion that the value of the top pick at only 10-15x the value of a 7^th round pick. The value of a pick has to be considered in relation to that of a team’s alternatives, in this case: undrafted free agents. Undrafted free agents are players eligible for the draft but not selected, whom any team can sign. Most teams pick up at least 5-10 of these players following the draft but they are not limited in this as they are with draft picks. The value of a late-seventh round pick, then, is only the excess it can be expected to provide over one of these players, which will be extremely low. The adjusted value for players to account for only the surplus also helps avoid assuming that ten players who deliver 1% of the salary cap in value are equal to one player who delivers 10% of the salary cap in value. Not only does the one player only take up one roster slot, but he will also be far more impactful on the field. You are unlikely to get the other ten on the field together and, if you could, they would only have room for one other player to deliver equal value to the ten who will play with the 10% player.

Relationship between Draft Performance and Team Performance

Now that a framework for evaluating performance of draft picks has been established, we can go back and examine how much the success of draft picks impacts the performance of teams.

To check how well this methodology works in the real world (or in Microsoft Excel), we will compare the relative draft performance of teams in terms of the total surplus value accrued (i.e., above what is expected for each draft pick) with the actual performance of teams on the field in subsequent seasons. The easiest benchmark to use here is whether a team made the playoffs or not. This has the distinct advantage of being a simple, binary outcome rather than widely distributed like winning percentage. Figure 8 demonstrates the relative draft performance of playoff and non-playoff teams in the preceding eight drafts.

Figure 8 – Relative draft performance for playoff (dotted line) and non-playoff (solid line) teams in the years before the playoff/non-playoff observation

Teams that made the playoffs in year 0 not only had better draft performance throughout the period, they outperformed the most in the five drafts from three to seven years prior – precisely the players who would likely be in their prime at year 0.

Moving on to our next figures we will take the average draft ranking for those five seasons (T-7 through T-3) and rank those averages relative to each other. The resulting data are summarized in figures 9 and 10. There is a relatively strong relationship (r-squared 0.55) between relative draft performance in terms of surplus career value for the five drafts three to seven years prior and percentage of teams making the playoffs in year 0. The relationship for total career value (as opposed to surplus career value) is slightly less strong with an r-squared of 0.46. The fact that the relationship is slightly stronger for surplus career value may be due to the fact that players with significant surplus value would be underpaid initially, or due to the scouting prowess of teams – a hidden variable that would also improve free agent signings and thus winning percentage. Both are relatively strong and we will continue to use expected career value as the baseline metric for our new Chart. The point here is to look at whether our definition of value and its calculation – in either form – explains something about team performance.

Figure 9 – Percent of teams making playoff by relative draft performance 3-7 years prior

Figure 10 – Percent of teams making conference championship game by relative draft performance 3-7 years prior

The data show that 59% of the teams in the top quartile (1-8) in this metric make the playoffs and 28% of them make the conference championship game. Only 21% of teams in the bottom quartile made the playoffs while only 3% of the bottom quartile teams made the conference championship game. Although the sample size is much smaller, the data also show that 72% of the Super Bowl winners for the period came from the top half of this metric.

Looking at several top performing teams of the past decade we can see this pattern in the real world. The Indianapolis Colts, for example, had 9 seasons in a row of being in the top quartile for the relative draft performance 3-7 years prior metric between 2001 and 2009. The Pittsburgh Steelers were in the top quartile every season through 2010 before drifting to 22^nd in 2011. They may be in a for several down years as their current rankings for future seasons, while still subject to change, are 26^th, 29^th and 27^th for 2012-2014. New England peaked at number 1 in this ranking in 2008, but their defense of an undefeated regular season was interrupted by Tom Brady’s knee injury. The New York Giants were perfectly average for the first 10 seasons of this ranking (the drafts from 1994 to 2003, ranks in 1997 to 2006) at an average of 16^th before improving to 8^th in their Super Bowl-winning 2007 season and dropping to three straight years of number 1 from 2009 to 2011. The counterexamples to these are the Packers and the Saints. The Packers are likely to improve as the rankings evolve because Aaron Rodgers lost out on several years due to sitting behind Brett Favre for two seasons. The Saints, on the other hand, have not ranked better than 7^th (in 2011) and ranked 15^th in their Super Bowl winning season. While their drafting has not been bad, they average 16^th in the last seven years, they owe their success in large part to picking up Drew Brees when he was coming off a major shoulder injury. This free agent signing was unique in that Brees was an established high-level QB who had already been to the Pro Bowl but his injury made signing him a high risk/high reward proposition.

Figure 11 – Number of players in the top 100 of career surplus value (value over expectation) from 1994-2011 by draft pick

The outperformance of teams that have a higher surplus value per pick highlights the value in the high standard deviation of top picks. Figure 11 demonstrates the draft picks of the top 100 players in terms of individual surplus career value (note that this accounts for the higher expectation of top picks as the expected career value is subtracted from each pick based on the relative draft position). Top picks are clearly the most likely to have a high career value surplus and teams that draft players with high career surpluses are likely to outperform on the field. This is why top picks are considered so valuable and why an analysis that shows picks at the beginning of the second round to be more valuable to teams may not be incorporating all of the impact of these players.

What does all this cost?

The cost of the number one pick under the new system is approximately 4.6% of the salary cap in average annual value. To compare this against the salaries for current players, in 2009 a salary of that level would have made Cam Newton or Andrew Luck somewhere in the range of the 21^st-highest paid QB in the league based on 2008 and 2009 salaries, comparable to such players as David Garrard, Derek Anderson, Chad Pennington or Tony Romo in those seasons.

Looking at where picks would be on the salary scale by position can be instructive in thinking about how teams make the actual player selection decision. If they are scouting a player who appears a lock to be a top defensive tackle and another who has a chance to be a pretty good defensive end, they may still be better served by picking the DE because of the high cost of finding a replacement at the position (Cam Newton’s salary would have been the 10^th-ranked DT but only the 21^st-ranked DE in 2008-2009.

For comparison with the previous system, Sam Bradford’s compensation would average 10.5% of the cap over the six years of his contract (not adjusting for changes in the cap, which has decreased from 2009). These numbers would make him the 5^th highest-paid QB in 2008 or the 7^th highest-paid in 2009, at a level with such players as Carson Palmer, Brett Favre or Matt Hasselbeck. Both totals put him well in front of where Cam Newton and Andrew Luck sit, while those and future number ones under the 2011 CBA have the advantage of being predictable – no more holdouts or 40% year over year increases on prior top picks.

There is also the fact that teams are simply not cap-constrained in most situations. In the run-up to the 2012 NFL Draft, only 3 teams were over the cap as of March 13^th and 17 teams had more than $10 million in available space[19]. Teams are also committed to spending a minimum proportion of the cap on players over various periods of the 2011 CBA. Faced with having to spend a certain amount on players teams may prefer spending on those with significant upside (rookies) versus those who are generally exposed only to downside in the form of skill erosion (veterans).

A key factor in the draft’s value, maybe the main one, is that the draft is the most reliable way to acquire players with a chance to significantly outperform their contract. Based on the structure of player movement in the NFL, the draft may be the only way to acquire elite players because their teams can prevent movement through the franchise tag. While there will still be many examples of players who underperform such as JaMarcus Russell or Vernon Gholston, teams wanting to acquire players like Peyton Manning or Julius Peppers simply will not find them at nearly the same concentration in later rounds or free agency. Peppers and Manning have notably changed teams as free agents in recent years, but both received extremely rich contracts and were not in the plans for their former teams – the Colts went on to draft new quarterback of the future Andrew Luck and the Panthers, after franchising Peppers one season, let him go and bottomed out at 2-14 in 2010 to earn the pick that became Cam Newton. 55% of all Pro Bowl seasons and 62% of all All-Pro seasons for players drafted from 1994 to 2011 were from players drafted in the first round.

While later picks may provide a more cost-effective unit of value, the absolute value over a replacement player drops off precipitously through the first round and with it the chance of getting a franchise-changing player. So even though value may be provided more efficiently per unit of cost by the first pick of the second round, it would take 2.7 of those players to equal the value of the first pick on the football field.

To put it another way, teams do not win by fielding a squad of average players. Super Bowl participants from 1994 to 2011 exemplify the importance of elite players. They average 1.97 first team All-Pros and 5.53 Pro Bowlers per participant – the averages from just the most recent decade (2002-2011) are very similar at 2.05 and 5.35[20]. If All-Pros and Pro Bowlers were evenly distributed, we would expect to find 0.75 and 2.38 per team, respectively. While scouting and player development have likely played a role in team construction, by working through the draft it would take roughly 51 average first rounder picks to yield the 1.97 first team All-Pros and 5.53 Pro Bowlers per season of an average Super Bowl participant. As the rounds get higher, the number of players to get those totals goes up dramatically to 185 in the second round, 375 in the third, 640 in the fourth and 825 in the fifth[21] in a league in which teams are limited to fielding a roster of 53 players, no more than 46 of whom are active[22].

There can be little doubt that taking a flyer on a player for a $5,000 signing bonus and a non-guarateed contract (roughly the contract terms for a seventh round pick) is cheaper per unit of value than guaranteeing $22 million to the top pick. The seventh rounder has only received only a very small amount of the guarantee of a top pick, but has an expected career value of roughly 1/16^th of the number one overall. Daryl Morey, writing in The Economist about the Houston Rockets’ failure to identify Jeremy Lin when they briefly had him under contract, makes this case more strongly than I can[23]. “With limited investment opportunities, teams are forced to choose only the players with the greatest likelihood of success, and then give them a long-term contract and a potential path to significant playing time.”[24] Morey is talking about basketball, with 15 players under team control against 5 starting positions, as compared to baseball, with hundreds of players under team control through the minor league system against 15 or so starting positions (including starting pitchers and sometimes designated hitters), but the concept applies similarly to football where the ratio is even lower. Teams have to fit 24 starters, assuming no one plays both ways and the punter and kicker are specialists, into an active roster of just 46 players.

Putting it all together

Now that we can see that elite draft picks matter to team success we are finally ready to put all the pieces together. The new Draft Value Chart (figure 12 and table 2) is composed based on a 253 pick draft, the number of picks in 2012, but the numbers and relative values can be adjusted to the actual number by defining the first value after the end of the draft as replacement level. Compared to the old Draft Value Chart, the new one produces a much shallower curve descending from the number one pick. The first 50% of value erodes by the 14^th pick in the new Chart, while it was the 7^th pick in the old Chart. The value descends to 25% of the number one pick by the 50^th pick in the new Chart while it was the 24^th pick in the old Chart. Because the value of the top several picks is reduced, the value of every other pick in the draft is higher relative to it. It is similar to the Massey-Thaler output in that it regards top picks as overvalued relative to later picks by the old Chart, but because the salary implications are considered to be secondary with no place but the draft to acquire these players and compensation brought under control by the 2011 CBA, teams are assumed to pick based purely on football value.

Figure 12 – New (solid line) and old (dotted line) Draft Value Chart

Table 2 – Revised Draft Value Chart with values expressed as a percentage of the first pick

2012 Draft

After declining precipitously in the 2000s, trades involving picks at the top of the draft were back in full force during the 2012 Draft. As noted earlier there were only four trades involving top five picks in the 2000s; the 2012 edition alone featured of those three trades. This is strong evidence that the 2011 CBA successfully addressed the salary conditions previously holding back teams trading for top picks.

Figure 13 – 2012 trades based on new Chart (Washington – St. Louis trade for the 2^nd overall pick is above the chart)

Figure 14 – 2012 trades based on old Chart (Washington – St. Louis trade for the 2^nd overall pick is above the chart

The trades of the 2012 Draft, shown in figures 13 and 14 in terms of % value of the first pick received by each team, appear to be moving slightly away from the values of the old Chart at least for those at the top.

The Rams’ trade of the #2 pick, and with it the right to pick Robert Griffin III, to the Redskins in exchange for the 6^th pick, the 39^th pick and two future first round picks is excluded on both graphs as it was a significant outlier. Both the new and old Chart suggest that the #2 pick the Redskins received has value equal to 87% of a first overall pick. In terms of compensation, the old Chart shows the Redskins gave up value equal to 149% of a first overall pick while the new Chart shows that the Redskins parted with 199% to acquire Griffin. In effect, RGIII would have to double the expected career value of the number one pick to match what the Redskins parted with to get him. The QBs to deliver this amount of value since the 1994 draft are Kerry Collins, Matt Hasselbeck and Steve McNair. This illustrates the uniqueness of picks at the top of the draft, where teams trading away understand that they are trading away specific players more than they are with later selections. The Rams knew they were sending RGIII to the Redskins, but Redskins did not know who they were sending back in return, they only knew the picks.

The trades involving the next three highest picks, the third overall (Minnesota traded to Cleveland), the fifth overall (Tampa Bay traded to Jacksonville) and the sixth overall (St. Louis traded to Dallas), show a deficit in terms of value received by the team trading the higher pick as calculated by the old Chart. Minnesota received 10% less than the Chart suggested, while Tampa Bay and St. Louis received 3% and 2% less, respectively. This divergence suggests that teams understand that the top picks are not accurately represented by the Chart currently in use.

Looking at the value of 2012 trades according to the new Chart (figure 13) the team trading away a higher pick received excess value in almost every trade. The practical limitation on taking advantage of this fact is a team’s ability to profitably utilize the excess picks that they get. For a team with many holes and many opportunities for rookies to play, trading down might help accumulate excess value that could actually be put to use.

How to Use This

The NFL has structured free agency to heavily weight the advantage to teams in keeping players they want. The salary cap annually forces decisions on second- and third-tier players, but it is exceedingly rare for a top-tier, blue chip player to change teams via free agency. Given the franchise tag and various other forms of control teams can also maintain a positive expected value in future contracts because of their leverage in negotiating. The draft remains the most reliable way to acquire elite players, the kinds of players who take a team to the playoffs and beyond, and the value of picks is in the career value they can provide.

The primary change in the new version of the draft value chart is that top picks are devalued somewhat relative to later picks. This may seem counterintuitive given the lengths to which the preceding sections have gone to demonstrate the value of top picks. That these picks are valuable is clear, but the value assigned to them in the original version of The Chart declined far too rapidly from the first several picks through the others. The revised version of The Chart shows that middle round picks are much more valuable than previously indicated relative to early picks, but it also shows that they are still much less valuable in terms of actual value (e.g., they may be twice as valuable relative to an early pick than they were previously, but still worth less than 10% of that pick). Attempts to value picks based on the value provided per unit of cost correctly show that the value per unit of cost is lower in the first round than in subsequent rounds. The alternative, however, is to pay a significantly higher premium for a free agent where the nature of competitive bidding will produce a winner whose estimate of value is most optimistic (or irrational) according to the Winner’s Curse in auctions. While the value per unit of cost is important, the fact that the revised CBA puts cost at a reasonable level for top picks clears up the problem that existed when top picks were paid like particularly overpaid free agents.

Previous attempts to reform the Draft Value Chart have not found wide acceptance, as demonstrated by the way picks still align with the old Chart as of 2012 (see figure 14). This is not to say that teams did not respond to the changes in value as top picks became ever more expensive, they simply stopped trading those picks because they could not agree on the revised value. Massey and Thaler noted that the emergence of the chart helped standardize values and increase the volume of trades. Because of the lack of opportunities to arbitrage, the value set by The Chart is extremely persistent but teams armed with the new version of The Chart teams can take several steps to take advantage of the error in valuation[25].

Teams with poor records who find themselves picking in the early portion of each round are likely to have a number of positions that they want to upgrade. These teams have an incentive to turn these picks into more picks, especially if they find a trading partner who bases their offer on the Old Chart. Trading down can yield additional expected value and teams with a number of holes are able to accommodate more new players, giving them a chance to demonstrate their ability where good teams may never get them enough playing time to see if they are good enough to develop into quality contributors. The key in the absence of this opportunity to trade down is to avoid trading up. Teams with many holes do not have the luxury of trading for that one missing player.

Teams with a better record, and corresponding lower pick, can make their choice based on their current roster situation. If they believe that there are a number of holes to fill, whether because of depth issues or impending player departures through retirement or free agency, they can pursue trading down. This strategy will be particularly powerful if a trading partner makes an offer based on the Old Chart so they will be overcompensated for moving down. If, however, they have few holes to fill on their team, then the strategic options are twofold: Trade up to acquire a single high-impact player or trade current picks for future year picks.

Trading up for a single high impact player is relatively risky, since top picks still have a reasonably high percentage of busts. If a team believes that a player has a relatively low floor, that he is unlikely to perform below an acceptable level, this can make trading up a better bet – but that requires a considerable amount of self-reflection on the certainty of a team’s scouting skill. The other risk is that teams will demand the values according to the Old Chart, meaning that the team trading up will be compelled to overpay significantly in terms of true expected value.

For future picks, the convention in the NFL is to value them one round worse than they are (i.e., a third round pick next year is worth only a fourth round pick this year)[26]. This convention makes trading current picks for future picks an especially good way for good teams to stockpile picks for when they will eventually need them. The fact that the Patriots are known for continually trading down to acquire additional picks, particularly in future drafts, is not likely to be a coincidence. As a team with many needs covered and few opportunities it is logically consistent that they do not need the same number of picks as an average team. Standing still and simply using the picks awarded by the league is always an option, but it is probably the optimal strategy only for a few of the 32 teams that are relatively well-balanced between pieces in place and needed players.

This analysis is just a taste of what is available to teams looking at data on production by draft pick. While it requires them to be realistic about their drafting ability, it can sharpen the behavior of even the best scouts and draft traders. Teams could look, for example, at a rank of the most draft pick/position combinations as a baseline (i.e., 1^st QB ranked #1, 2^nd QB #2, 1^st Tackle #3 and so on) and make adjustments from there where certain players appear to be above or below the expected production. They could then update these values with each passing season to reflect the way position salaries are changing. This analysis could be expanded to include a detailed look at free agency and identify the positions at which players who still have value are commonly available. In the meantime, hopefully some teams will take advantage of the inefficiencies of The Chart and give this new one a spin.

And what about the Cowboys, the team that gave us the original Chart and used it to help build their early-90s dynasty? There were a few good drafts in the first couple salary cap years, but their draft performance (in terms of career value surplus from the drafts 3-7 years prior) didn’t crack the top quartile until 2008. They remained in the top quartile for the entire 2008-2011 period in which they twice made the playoffs and averaged 8.5 wins per season. The future, however, does not look so great: the current projection of the 2012-2014 rankings are 20, 26 and 24[27]. It may be a while before the Cowboys put together their next dynasty.

---

[1]http://webcache.googleusercontent.com/search?q=cache:52IzoRlbX1gJ:cowboyszone.com/forums/showthread.php%3Ft%3D22+mike+mccoy+%22dallas+morning+news%22+2004&cd=2&hl=en&ct=clnk&gl=us&client=firefox-a

[2] http://sports.espn.go.com/nfl/draft06/news/story?id=2410670

[3] http://faculty.som.yale.edu/CadeMassey/documents/loserscurse_new.pdf

[4] New York Times archives, various

[5] http://thebiglead.com/index.php/2012/04/25/nfl-draft-will-the-new-rookie-wage-scale-system-lead-to-more-trades/

[6] http://profootballtalk.nbcsports.com/2011/08/04/last-three-first-round-holdouts-about-money-not-fourth-year-guarantee/ (Updated to reflect that the guarantee was an issue)

[7]http://sportsillustrated.cnn.com/2012/writers/don_banks/04/26/2012.nfl.draft.first.round/index.html?eref=sihp&sct=hp_t12_a2

[8] http://faculty.som.yale.edu/CadeMassey/documents/loserscurse_new.pdf

[9] http://mlb.mlb.com/pa/info/faq.jsp

[10] http://www.cbafaq.com/salarycap.htm

[11] http://espn.go.com/nfl/story/_/id/7820396/nfl-restricted-free-agency-verge-extinction

[12] http://nfllabor.files.wordpress.com/2010/01/collective-bargaining-agreement-2011-2020.pdf

[13] USA Today Salary Database, salary cap numbers used (as opposed to actual cash in a given year)

[14] Pro-Football-Reference.com, punters and kickers excluded

[15] In 2011, two of the top five players in receiving yards were tight ends, Jimmy Graham and Rob Gronkowski, and the teams for which they played, New Orleans and New England, were two of the top three offenses in points and yards per passing attempt. The 2008-2010 seasons featured one tight end each year in the top five receivers with 2005-2007 having none.

[16] As part of the 2011 CBA, teams also hold explicit options for the fifth season of all first round picks. Players drafted in the top ten have an option that gives the team the right to the fifth season at a cost of the average salary of the top ten players at that position. Players drafted in the rest of the first round have an option for the fifth season at a cost of the 3-25 salaries for that position. Since both the asset price (average value for that draft pick) and strike price (average of either top ten or 3-25 salaries by position, weighted by actual occurrence of each position) are in inflation-proof terms (% of the salary cap) we don’t need to price anything in for the interest rate aspect of the option. Because the seasonal volatility is a better proxy for only the fifth year, but declining the option may be considered to lose out on the future implicit options on the player’s career, we’ll split the difference and use the average of the career and average season volatility numbers in the calculation. For the top ten picks the value of the option declines rapidly, matching the trajectory of career value, from roughly 0.5% of the salary cap ($600k in 2011) for the number one pick to 0.21% of the salary cap for picks 8-10. At pick number 11 the value jumps up by 0.1% to 0.31% of the salary cap, worth an additional $120k in 2011, before descending gently to 0.16% value for the 32^nd pick.

[17] There are some obvious limitations with looking at data from the free agency period only. For starters, many of the players drafted are still in the league and running up additional seasons and statistics. The entire 1994 draft class has retired but 36 players drafted from 1995 to 1999 played in the 2011 season. The effect of all this is that the statistics that define the value of a drafted player (career seasons played, All-Pro and Pro Bowl seasons per career, seasons as starter) are all understated. Relative to each other, however, they are equally understated. In the 18 seasons in the sample from 1994 to 2011, the range of total picks in the draft is only 222 to 262, and the second lowest total is 240 so the top 222 picks have the same number of observations. The two alternatives to this method are using only those drafts, like 1994, from which all of the drafted players have retired or creating an estimation model to simulate the remainder of an active player’s career. Looking only at older drafts creates a data set that does not resemble the present in terms of scouting technologies, playing styles at various positions and a host of other factors. Applying some sort of estimation model compounds the problem of missing data by adding the problem of another potentially biased or incorrect calculation. As long as the impact of this limitation in the depression of career statistics is understood, the analysis should benefit from the inclusion of additional data.

[18] The numbers quoted are for the first, second, third and so on sets of 32 picks. The actual tenure by round would not be comparable because the number of picks per round has changed from 1994 to 2011 as several teams were added to the league. These averages are subject to the same limitations noted on the previous footnote as recent drafts feature numerous players whose careers are not yet complete.

[19] http://blogs.nfl.com/2012/03/13/salary-cap-numbers-for-all-32-nfl-teams/

[20] Pro-football-reference.com

[21] Picks to equal average participant equal the average of the picks required for All-Pro and Pro Bowl

[22] http://nfllabor.files.wordpress.com/2010/01/collective-bargaining-agreement-2011-2020.pdf

[23] Jeremy Lin is a basketball player whose 2012 emergence for the New York Knicks made international news due to, among other things such as his degree from Harvard and being the first American NBA player of Chinese or Taiwanese descent, the fact that he was undrafted and had been cut from two different teams before catching on with the Knicks. He subsequently signed with the Houston Rockets as a free agent in the summer of 2012, giving Daryl Morey one more shot to get it right. Source: http://www.nytimes.com/2011/12/29/sports/basketball/jeremy-lin-knicks-newest-addition-is-out-to-prove-hes-not-just-a-novelty.html?_r=1

[24] http://www.economist.com/blogs/gametheory/2012/04/predicting-performance-basketball

[25] http://faculty.som.yale.edu/CadeMassey/documents/loserscurse_new.pdf

[26] http://faculty.som.yale.edu/CadeMassey/documents/loserscurse_new.pdf Massey and Thaler note that the rule of thumb for future picks is to devalue them by one round. This is a significant opportunity at the top of the draft as it massively undervalues picks. As an example, by accepting a 1^st round pick for a 2^nd round pick a team will send away 25.7% of the cap in career value and receive 47.1% - an 83% appreciation in exchange for waiting just one year if we assume middle-of-the-round values. Of course, stable teams can take advantage of this because management of unstable teams with a high probability of being fired without a winning season may have a discount rate higher than 83% - they won’t be around unless they win so who cares what’s left. A strong owner is the best possible line of defense against this because they have the furthest possible time horizon among a team’s leadership structure.

[27] Because many players from the drafts that make up those rankings are still playing and early in their careers, there is still hope for the Cowboys to develop players like Tyron Smith, Dez Bryant and many others into stars and move up the rankings