Luck vs Skill - Are picks after trading up better than others?

As some of you have no doubt noticed I have spent my last few posts looking at the level of skill involved in the NFL draft. Whether it’s year-over-year or individual selections in the same year, I have had no luck finding an outcome in which skill appeared to play a significant role.  

For my third cut at the subject I am trying a narrower focus in search of skill: draft day trades. Specifically, I will be looking at the highest pick involved in each draft day trade. These trades ought to show us the picks about which decision makers were most confident. They had to offer enough compensation to the pick’s owner to induce them to trade and they did it with a specific player in mind. Though these trades include all transactions on the day of the draft, a great majority of them were consummated during the selection window, only when that pick granted the right to a specific player the team trading up desired.

Thanks to the fantastic website www.prosportstransactions.com I was able to work quickly through the data collection phase and identify those trades that took place on draft days from 1994 to 2006, the same period as my previous post. After working through the details[1] we are ready to go.

That value chart is irrelevant because teams are trading for specific players

 
Long story short: there is still nothing to see here. Out of 245 picks in the data set, 108 were “good” according to the value expected by the Sports + Numbers draft value chart. Given the distribution of picks we would have expected 104. Not a good sign to start. 

The individual p-values are nothing remarkable and those of rounds 3 through 5, flirting with a whole 50% chance of non-random difference from average, could be substantially undone by a single additional bad pick, their values retreating to 83%, 79% and 77%, respectively. These are not robust findings of skillful GMs swindling their dimmer peers for an overlooked gem.  

One of the more common retorts of those confronted with a trade that looks unfavorable according to the draft value chart (any of the draft value charts) is that teams are trading for specific players rather than some nebulous conception of value – so the trade can’t be evaluated by those fancy charts. The results of this analysis directly contradict that interpretation of trades because the players selected after trades appear to be as successful (or unsuccessful) in the league as those selected in the normal course of the draft. Therefore, the value chart seems like a pretty good way to deal with trade evaluation.  

Top picks too  

A few peeks at the early first round trade up picks might help head off some of the “..but what about ____?” comments.

Cutting up expected success rates by round is somewhat arbitrary, as I noted in my earlier post, so would it make a difference if we just look at the really high picks? The overall first round expected success is 51.6% while the top 10, including both traded and non-traded picks, comes in at 50.8% even against much higher expected value (see all the details here).  

The 17 top 10 picks that were traded only feature 7 (41%) that turned out to be good picks. While some of the good ones were very good (Eli Manning, Walter Jones, Champ Bailey) some of the bad ones were very bad (Ki-Jana Carter, Trev Alberts, Jonathan Sullivan).

Expanding out to the full first round for a bigger sample, the average surplus of the trade up group of is 5.08% while the non-trade up group comes in at 4.21%[2]. This difference in performance is worth just over $1 million in career salary cap value. It’s not nothing, but it’s not much spread over the average 8+ year careers of these players (some of whom are still playing). In terms of draft picks it is worth something like the 240th pick, a mid-7th rounder.


[1] I included the top pick involved in picks for picks transactions or players and picks for picks transactions, provided the side with a player on it did not have the highest pick among those transacted. I also included pre-draft trades of the number one pick – and mid-draft trades of Eli Manning – because the team selecting first gets their choice of player and I assumed that they had one in mind. The higher pick is the only one included in the data set. The evaluation of a “good” pick depends only on the production of that pick relative to the expectation and not on the players eventually selected with whatever was traded away. That is more than enough fine print for one post.

[2] The difference is positive for both because the last several years of drafts are not included, where players have had shorter careers that depress the total value in the full set. I’m working on some revisions to my value chart that will take the growth curve into account and get at a better estimate of total career value.