[Fair warning: what follows is quite lengthy]
Well, it is performance review time at work and this reminded me of a post I’ve been meaning to write for a while.
An issue that has always interested me is how organizations measure individual performance. Organizations have finite resources and therefore must deploy those resources in the most efficient manner, maximizing their value. Given the large percentage of resources invested in personnel, organizations have a huge incentive to get those investments right. However, calculating an accurate ROI on employees is probably one of the hardest things to do. To explore why–and how it might be done better–I turn to the world of sports, baseball in particular.
There has long been a debate within the Sabermetric community (and between purists and Sabermetricians) regarding the statistical relevance of “clutch”: the ability of a player to elevate their performance in key situations in a manner significantly different from their performance in normal situations. Early research by such pioneers as Bill James found that the attribute of clutch didn’t exist–much like the idea of a “hot-hand” in basketball, the appearance of a clutch performance (e.g. a usually mediocre batter managing to hit .500 in a playoff series) was nothing more than a statistical artifact. If you were to look at any 5-7 game stretch during the 162-game regular season you are just as likely to find even average hitters going 4 for 8 or 8 for 16 as you would be to find them going 2-8 or 0-16. Over a long enough time period, these streaks even out and players regress to their mean performance. Basically, if you are a .250 hitter, over a long enough period of time your performance will return to its mean, despite occasional swings to the extreme left and right sides of the bell curve.
More recent studies have looked to expand upon earlier research and refine how we search for clutch performances. A common way to do this is to not look at single games that were more important (e.g. post-season play), but rather particular moments that alter the probability of a team winning that particular game. This approach has been termed “leverage“: Continue reading