Sunk cost and the NFL Draft

I’ve looked at the NFL Draft a lot since starting this blog. As the draft was here in Chicago this year, I found myself running into a number of jerseys on the street when I went out for lunch on Thursday and Friday. Even more surprising than the fact that people had travelled – in some cases from pretty far away according to the jerseys – was the fact that a lot of them were wearing jerseys of players who were disappointments if not outright busts. It got me thinking about sunk costs and whether teams are any better than their fans about cutting their losses.

To try to get at this we’ll need to know how much teams value their draft picks – conveniently we do know this via the Jimmy Johnson-popularized draft value chart – and then compare this to how much those players are used. Usage is a bit tricky but I’m going to approximate it with games started (1 full game) plus games played (2014 avg snaps non-starter / 2014 avg snaps starter, by position).

Before even getting to questions of usage, there is a significant disparity in the proportion of players from each round who end up making a roster.

Round	% on Roster Year 1
1	97%
2	94%
3	83%
4	81%
5	70%
6	62%
7	52%

I am guessing that most of this comes down to talent disparity, but there is certainly some aspect of sunk cost at work here. Lots of later round picks – to say nothing of undrafted players – never make it onto a roster to get into the rest of this analysis. They are, however, not the topic of this analysis. I want to see if a player’s draft value still impacts playing time even after making a roster.

The first cut of this is simply to look at draft weight and usage, checking how much the former impacts the latter. The regressions for each of a player’s first 6 seasons are below:

Usage vs Draft Weight
			Draft Weight
Year	R^2	Intercept	Coefficient	P-Value
1	0.22	4.53	15.87	0.00
2	0.16	6.88	13.87	0.00
3	0.10	8.10	10.71	0.00
4	0.08	8.84	9.18	0.00
5	0.05	9.49	6.58	0.00
6	0.04	9.88	6.14	0.00

The draft weight is a significant variable throughout the first 6 years of a player’s career, but the strength of that relationship declines over time. The 1^st year model explains 22% of the variation in usage while the 6^th year model explains just 4%.

Of course, this is highly stylized. While coaches will always have more information than is reflected in statistics, there is some additional quantitative data that can be added to the model. For the next round we’ll add past performance, represented in terms of the square root of a player’s AV over the past 3 seasons. The 3 is a bit arbitrary, but bear with me. The square root allows us to make sure additional performance has diminishing returns in predicting usage in year N. If, for example, a player is an all-pro for years N-3 to N-1, he can still only play 16 games in year N. On the other hand, if a player has been injured for a year or two, a solid year will still have a significant impact on predicted usage.

Usage vs Draft Weight + SQRT Prior 3 Years AV
			Draft Weight		SQRT - Prior 3 Yrs AV
Year	R^2	Intercept	Coefficient	P-Value	Coefficient	P-Value
1	0.22	4.53	15.87	0.00	N/A	N/A
2	0.29	4.89	7.28	0.00	2.29	0.00
3	0.29	4.11	2.46	0.00	2.26	0.00
4	0.26	3.77	0.97	0.13	2.00	0.00
5	0.26	3.40	-0.78	0.24	2.02	0.00
6	0.25	3.45	0.24	0.74	1.93	0.00

Lots of stuff to unpack here. First, the R-squared stays roughly the same throughout the 6 seasons. The draft weight, however, drops rapidly with the coefficient moving toward 0 and the p-value coming up past the 0.10 threshold in years 4-6. The square root of the prior 3 years AV stays pretty consistent around 2 for the coefficient and the p-value is pegged at 0 throughout, indicating sustained significance.

It certainly isn’t controversial that recent past performance has a strong effect, but it is a bit surprising to see draft weight have an impact well into a player’s career. With one last set of regressions we’ll try to get a bit of insight into a player’s future skill level by looking at actual future performance. The methodology is the mirror image of our variable for past performance as we’ll consider the square root of AV in the next 3 seasons (e.g., years 2-4 for a rookie). This should serve as our proxy for how a coach with perfect foresight will evaluate a player. If they can make a significant impact in one of the next three years, or a moderate impact in all three, we would expect to see that reflected in present year usage regardless of track record.

Usage vs Draft Weight + SQRT Prior 3 Years AV + SQRT Next 3 Years AV
			Draft Weight		SQRT - Prior 3 Yrs AV		SQRT - Next 3 Yrs AV
Year	R^2	Intercept	Coefficient	P-Value	Coefficient	P-Value	Coefficient	P-Value
1	0.35	2.86	9.14	0.00	N/A	N/A	1.04	0.00
2	0.41	3.52	3.89	0.00	1.46	0.00	1.11	0.00
3	0.43	3.16	0.47	0.36	1.32	0.00	1.24	0.00
4	0.41	3.56	-0.14	0.80	0.98	0.00	1.29	0.00
5	0.41	3.68	-1.61	0.01	1.00	0.00	1.28	0.00
6	0.38	3.99	-0.60	0.35	1.00	0.00	1.17	0.00

The overall model is improved from ~0.22-0.29 R-squared to ~0.35-0.43. With respect to draft weight, year one still shows a significant impact, though the coefficient is reduced from the prior models, and year 2 holds up as well before years 3-6 drop to insignificance (I’ll put my money on the 1% chance that the significance of year 5 is random). At this point draft weight appears to be more a proxy for prior performance, as it should be, than a lingering sunk cost impacting decision making.

My next post will look at the first couple years in a bit more detail, to try to tease out whether relative usage is due to draft weight or other factors.

What’s Missing?

The future performance serves here as a proxy for coach’s judgment/underlying ‘true’ skill level, but there are a couple other factors missing in this analysis. The two main ones are the particular situation – whether a player is sitting behind a veteran at the same position – and injuries. Both can have a big impact on playing time but represent a bit of a modeling challenge. Feel free to use the comments if you have any ideas for incorporating these into the model. As it stands, I’m pretty happy with a model that has ~0.4 R-squared (~0.25-0.30 with only data available prior to the season).