≡ Menu

The Stats Say: Popular opinion (ESPN’s NBA Rank) badly misjudges several Rockets

NBA rank vs RAPM chart

There were two juicy datasets crying out for comparison:

  1. ESPN’s/TrueHoop’s NBA Rank offers a fascinating window into public perception. Every active NBA player is rated on a 0-to-10 scale of the player’s expected “overall level of play for the upcoming season.” The ratings are submitted by ESPN’s Forecast Panel, which currently includes 215 commentators, analysts, geeks, and writers/bloggers (including our own Rahat Huq and Michael Pina).
  2. Jeremias Engelmann’s RAPM statistic offers an objective measure of every player’s overall value. There are other measures that do the same (e.g., PER, WP48), but they are based on box score statistics and so don’t account for a player’s defensive contributions nearly as well as RAPM. RAPM takes the simple +/- statistic and then does some fancy Bayesian stuff in order to isolate the contribution of an individual player from the contributions of his teammates and the quality of his opponents, thus giving a hopefully-unbiased picture of the degree to which a player is helping or hurting his team when he is on the court.

The full set of results is shown in the table below and the two highlighted columns are presented in the chart above. It looks like NBA Rank does a generally good job but misses pretty badly on several Rockets (assuming we can trust RAPM as an unbiased measure). Most interesting to me is Omer Asik’s severe undervaluation. This post is intended for results rather than discussion but I hope to pick up this topic in a later post.

Note on the methods: To make the two sets of statistics comparable, I had to create the “RAPM rating.” I did this by ranking every player based on their RAPM from last season and then I pulled the corresponding rating from NBA Rank. For example, James Harden’s RAPM ranked 13th in the league, so I looked at the 13th ranked player according to NBA Rank (Paul George) and assigned Harden that rating (8.39). It isn’t a perfect method, but it at least gives an interesting comparison.

NBA rank vs RAPM table

One frustrating thing about NBA Rank is that it flattens out the wildly varying talent pool. For example, based on the NBA Rank ratings you might assume that LeBron is about 10-15% more valuable than James Harden or Dwight Howard. The chart below is a sobering reminder of how untrue that is (or at least was last year).

RAPM vs PER

All this talk about players’ rank and value seems to be missing one important thing: dollars. I think the definition of a player’s true value is how much they would be paid in the open market if we lived in a perfect universe where general managers knew everything and did not act stupidly. Actual salaries aren’t a very good indicator of true value because (1) general managers do act stupidly (see Joe Johnson’s contract) and (2) we don’t have a truly open market with rookie contracts, veteran minimums, salary caps, and other constraints imposed by the owners and NBPA.

The chart below shows my estimates of the Rockets’ true value (i.e., “deserved salary”) compared to their current actual salary. (Note that Jeremy Lin and Omer Asik are scheduled to make $5M this year and $15 next year, so I took the average of the two.) To estimate deserved salaries, I first plotted actual salaries by last year’s RAPM and then drew a best-fit exponential line through the data. I then fiddled with the equation so that it estimated that Dwight Howard was paid exactly fairly. That’s just an assumption I was making to get reasonable-looking estimates.

current salary vs deserved salary

While Howard and Harden might each be worth about $18-$21 million per year in the open market, when the same deserved-salary equation is applied to LeBron, he is estimated to earn/deserve $90 million per year(!). That might be an overestimate, but then again it might not. Chris Paul and Kevin Durant are estimated to earn/deserve $49 million and $46 million, respectively. Even if those are overestimates, it goes to show just how wildly varied are the differences in talent level and value among even the league’s top several players.

There are many implications and so many directions I could go with this, but I will hold my tongue for now.

View this discussion from the forum.

in essays

{ 0 comments… add one }

Login to leave a comment.
Total comments: 20
  • 2016Champions says 12 months ago

    Don't get me wrong, no one is saying RAPM is perfect, but it should definitely replace on/off court numbers and +/-

  • 2016Champions says 12 months ago

    Cool. That's why I asked. I figured it must be something about sample size. As far as Camby's defense, no one believes that even in limited minutes he is capable of playing the level of defense he played in his DPoY year, do they? Anyway, it's an interesting measure, but like all basketball stat systems it's still quite flawed. No one has quite nailed how to quantify basketball well yet. At least not compared to baseball.

    I'm not sure why your implying Camby's DRAPM is DPoY caliber when it's 1.8 because in contrast Dwight's logged over 7 more than once, and all the other elite defensive big men hover around 5 or 6. So while 1.8 is nothing to scoff at for a 38 year old, it's not elite. The last time Camby had over 6 was in 2010, and his defense has been on a steady decline since then.

  • vonsteve says 12 months ago

    Yes, you a missing a couple of things:

    1. He's a former DPoY who is still capable of playing elite defense in limited minutes.

    2. The creator of RAPM stated that a minimum of 60 games should be used because this stat works best with a large sample size. 60 games of starter minutes (lets say 30 minutes) would be at least 1800 minutes, and Camby played only 250 minutes last season. Small sample sizes exaggerate results.

    Cool. That's why I asked. I figured it must be something about sample size. As far as Camby's defense, no one believes that even in limited minutes he is capable of playing the level of defense he played in his DPoY year, do they? Anyway, it's an interesting measure, but like all basketball stat systems it's still quite flawed. No one has quite nailed how to quantify basketball well yet. At least not compared to baseball.

  • 2016Champions says 12 months ago

    Here's an old article which used the on/off court numbers, and honestly this is something I've come across a lot:http://www.slcdunk.com/2013/2/6/3956192/all-the-reasons-that-paul-millsap-is-still-the-jazzs-starting-power

    Now that RAPM is becoming more recognized, we can stop using the inferior +/- or on/off court stats and replace them with RAPM. The other option is to just stop using stats all-together just because they're flawed in one way or another, but that's just silly.

  • thejohnnygold says 12 months ago

    You're been pretty touchy lately, just keep calm and love Houston Rockets!

    Like I said, RAPM is just an improved version of the on/off court stats commonly seen on scouting reports. You don't have to like it but it seems very silly to rant against it lol

    Maybe I have been touchy lately. My bad. I don't take offense to RAPM (despite my strong dislike for it)--it was the the other part that went with it that got under my skin.

  • 2016Champions says 12 months ago

    Look, I know RAPM is not going away, but can we not speak of it as if those who dislike it "just don't get it" or don't understand, or whatever. I DO get it. Which is why I like it just as much as any other statistic that is fabricated, twisted, manipulated, misused, and ultimately, despite only existing to do the opposite, manages to not account for what actually happens on the court. It's fine if people want to discuss it and play with it, but let's not place the pedestal too high here. At this point, it's the equivalent of a neanderthal using a stone to bash things vs. one who has learned to tie that stone to the end of a stick. Yeah, that's better...but you're still in the stone age, buddy.

    You know what a good stat would be to go with DRAPM? A stat called "My-bad's". It's a stat, like rebounds or assists, that you earn for being the person responsible for the other team scoring--Not necessarily the guy defending the one who scored. What about the guy who missed the back door cutter which forced a rotation allowing that guy's man to score. Ridiculous shots that just get hoisted on a prayer shouldn't count either (think Monta Ellis). The guy who falls down opening up a free lane to the basket. The center who can't keep his arms straight and fouls every guard that penetrates the lane....there is so much that is missed.

    For Houston last season, guys like D-Mo, T-Jones, Harden, and G. Smith would be league leaders in "My-bad's". Those deficiencies are too great to overcome because one of those guys is always on the court. What does that do for the other guy's DRAPM? Nothing good. The fact that Asik still mustered a solid score says everything. He is so very, very under-rated.

    Sorry for the rant. I don't mind people using RAPM stats and I found the original post interesting.

    You're been pretty touchy lately, just keep calm and love Houston Rockets!

    Like I said, RAPM is just an improved version of the on/off court stats commonly seen on scouting reports. You don't have to like it but it seems very silly to rant against it lol

  • 2016Champions says 12 months ago

    Am I the only one who noticed Camby? RAPM has his value higher than Parsons. Am I missing something?

    Yes, you a missing a couple of things:

    1. He's a former DPoY who is still capable of playing elite defense in limited minutes.

    2. The creator of RAPM stated that a minimum of 60 games should be used because this stat works best with a large sample size. 60 games of starter minutes (lets say 30 minutes) would be at least 1800 minutes, and Camby played only 250 minutes last season. Small sample sizes exaggerate results.

  • thejohnnygold says 12 months ago

    ??????????? did I miss something or was this meant for another subject?

    No, he's referring to the chart above saying Lin's performance only warranted roughly $2.5M last season. Not sure who Alex is. I think he is misunderstanding that this was done objectively, through performance-based criteria, and nobody went out of their way to say Lin isn't worth the money. (I will say, the criteria used leaves a great deal to be desired)

  • rockets best fan says 12 months ago

    You are out of your mind to say that Jeremy Lin only deserves $2.5m after all the things he did.
    Tell it to Alex. He only thinks that Lin makes the days of the Rockets. Without Lin, they couldn't possibly be in this position now. They wouldn't be in the playoff last year. They would have fallen apart. He deserves every bucks he's receiving. Once his contract expires, you will see his true value in the market; that is much more than he's getting now.

    ??????????? did I miss something or was this meant for another subject?

  • myjohnlai says 12 months ago

    You are out of your mind to say that Jeremy Lin only deserves $2.5m after all the things he did.
    Tell it to Alex. He only thinks that Lin makes the days of the Rockets. Without Lin, they couldn't possibly be in this position now. They wouldn't be in the playoff last year. They would have fallen apart. He deserves every bucks he's receiving. Once his contract expires, you will see his true value in the market; that is much more than he's getting now.

  • timetodienow1234567 says 12 months ago Maybe RAPM is not that great...
  • vonsteve says 12 months ago

    Am I the only one who noticed Camby? RAPM has his value higher than Parsons. Am I missing something?

  • rockets best fan says 12 months ago

    so would you trade Harden and Howard for LeBron?

    Nope. LBJ is a stud, but 2 stars for one isn't an upgrade to me. both Howard and Harden are top ten players.

  • Cooper says 12 months ago

    so would you trade Harden and Howard for LeBron?

    Hypothetically yes if he was going to re sign
  • thejohnnygold says 12 months ago

    Look, I know RAPM is not going away, but can we not speak of it as if those who dislike it "just don't get it" or don't understand, or whatever. I DO get it. Which is why I like it just as much as any other statistic that is fabricated, twisted, manipulated, misused, and ultimately, despite only existing to do the opposite, manages to not account for what actually happens on the court. It's fine if people want to discuss it and play with it, but let's not place the pedestal too high here. At this point, it's the equivalent of a neanderthal using a stone to bash things vs. one who has learned to tie that stone to the end of a stick. Yeah, that's better...but you're still in the stone age, buddy.

    You know what a good stat would be to go with DRAPM? A stat called "My-bad's". It's a stat, like rebounds or assists, that you earn for being the person responsible for the other team scoring--Not necessarily the guy defending the one who scored. What about the guy who missed the back door cutter which forced a rotation allowing that guy's man to score. Ridiculous shots that just get hoisted on a prayer shouldn't count either (think Monta Ellis). The guy who falls down opening up a free lane to the basket. The center who can't keep his arms straight and fouls every guard that penetrates the lane....there is so much that is missed.

    For Houston last season, guys like D-Mo, T-Jones, Harden, and G. Smith would be league leaders in "My-bad's". Those deficiencies are too great to overcome because one of those guys is always on the court. What does that do for the other guy's DRAPM? Nothing good. The fact that Asik still mustered a solid score says everything. He is so very, very under-rated.

    Sorry for the rant. I don't mind people using RAPM stats and I found the original post interesting.

  • Johnny Rocket says 12 months ago

    To defend the ESPN rankings for a moment: they are about predicted performance for the coming year, not an assessment of past years. Somebody could reasonably argue that Chandler Parsons will do better this year (on a per minute basis) with Howard in the line up because there will more open looks for corner threes, more easy cuts to the basket, and more fast-break layups and dunks. Conversley, one should undervalue Asik relative to last year's RAPM because there is a high probability that Asik will play a lot less. Even if I think that Asik is really the 18th best player in the league, it makes sense to devalue him if he is only playing 20 minutes per game.

    Put it this way: do you really think that Marcus Camby is the 127th best player in the league? I doubt it, but those are the results you sometimes get with stats calculated on a per-minute basis.

  • timetodienow1234567 says 12 months ago The salaries don't match up.
  • John P says 12 months ago

    so would you trade Harden and Howard for LeBron?

  • 2016Champions says 12 months ago

    I like that more people are talking about RAPM. People make the mistake of using it as a ranking system, but eventually people will start to understand it and just see it for what it really is--an improved alternative to the on/off court numbers commonly seen on scouting reports and www.82games.com

  • timetodienow1234567 says 12 months ago Of course that's assuming RAPM is an actual measure of worth. Interesting article though.

Leave a Comment