≡ Menu

The Stats Say: Popular opinion (ESPN’s NBA Rank) badly misjudges several Rockets

NBA rank vs RAPM chart

There were two juicy datasets crying out for comparison:

  1. ESPN’s/TrueHoop’s NBA Rank offers a fascinating window into public perception. Every active NBA player is rated on a 0-to-10 scale of the player’s expected “overall level of play for the upcoming season.” The ratings are submitted by ESPN’s Forecast Panel, which currently includes 215 commentators, analysts, geeks, and writers/bloggers (including our own Rahat Huq and Michael Pina).
  2. Jeremias Engelmann’s RAPM statistic offers an objective measure of every player’s overall value. There are other measures that do the same (e.g., PER, WP48), but they are based on box score statistics and so don’t account for a player’s defensive contributions nearly as well as RAPM. RAPM takes the simple +/- statistic and then does some fancy Bayesian stuff in order to isolate the contribution of an individual player from the contributions of his teammates and the quality of his opponents, thus giving a hopefully-unbiased picture of the degree to which a player is helping or hurting his team when he is on the court.

The full set of results is shown in the table below and the two highlighted columns are presented in the chart above. It looks like NBA Rank does a generally good job but misses pretty badly on several Rockets (assuming we can trust RAPM as an unbiased measure). Most interesting to me is Omer Asik’s severe undervaluation. This post is intended for results rather than discussion but I hope to pick up this topic in a later post.

Note on the methods: To make the two sets of statistics comparable, I had to create the “RAPM rating.” I did this by ranking every player based on their RAPM from last season and then I pulled the corresponding rating from NBA Rank. For example, James Harden’s RAPM ranked 13th in the league, so I looked at the 13th ranked player according to NBA Rank (Paul George) and assigned Harden that rating (8.39). It isn’t a perfect method, but it at least gives an interesting comparison.

NBA rank vs RAPM table

One frustrating thing about NBA Rank is that it flattens out the wildly varying talent pool. For example, based on the NBA Rank ratings you might assume that LeBron is about 10-15% more valuable than James Harden or Dwight Howard. The chart below is a sobering reminder of how untrue that is (or at least was last year).

RAPM vs PER

All this talk about players’ rank and value seems to be missing one important thing: dollars. I think the definition of a player’s true value is how much they would be paid in the open market if we lived in a perfect universe where general managers knew everything and did not act stupidly. Actual salaries aren’t a very good indicator of true value because (1) general managers do act stupidly (see Joe Johnson’s contract) and (2) we don’t have a truly open market with rookie contracts, veteran minimums, salary caps, and other constraints imposed by the owners and NBPA.

The chart below shows my estimates of the Rockets’ true value (i.e., “deserved salary”) compared to their current actual salary. (Note that Jeremy Lin and Omer Asik are scheduled to make $5M this year and $15 next year, so I took the average of the two.) To estimate deserved salaries, I first plotted actual salaries by last year’s RAPM and then drew a best-fit exponential line through the data. I then fiddled with the equation so that it estimated that Dwight Howard was paid exactly fairly. That’s just an assumption I was making to get reasonable-looking estimates.

current salary vs deserved salary

While Howard and Harden might each be worth about $18-$21 million per year in the open market, when the same deserved-salary equation is applied to LeBron, he is estimated to earn/deserve $90 million per year(!). That might be an overestimate, but then again it might not. Chris Paul and Kevin Durant are estimated to earn/deserve $49 million and $46 million, respectively. Even if those are overestimates, it goes to show just how wildly varied are the differences in talent level and value among even the league’s top several players.

There are many implications and so many directions I could go with this, but I will hold my tongue for now.

View this discussion from the forum.

in essays