, , ,

The NY Times had a great article yesterday profiling the increasing fortunes for advanced statisticians.  As the world has become more data-driven and flush with raw numbers, the need to derive sophisticated insights from all that data has increased. Data does not speak for itself:

The new breed of statisticians tackle that problem. They use powerful computers and sophisticated mathematical models to hunt for meaningful patterns and insights in vast troves of data. The applications are as diverse as improving Internet search and online advertising, culling gene sequencing information for cancer research and analyzing sensor and location data to optimize the handling of food shipments.

With the rise in data also comes the opportunity to extract profits if one can identify the right insights and patterns.  For example, I.B.M. recently launched a new group that will focus on business analytics and optimization.  They plan to grow the group aggressively.

With this shift towards a data-driven world has come a corresponding shift in the value of certain skills.  In this case, sophisticated statisticians and the analytically-minded find themselves in a position where their skills now command both respect and high salaries.  It has also allowed people to pursue careers that weren’t necessarily available to them even just a few years ago. Two of my favorite examples are the rise of statisticians in professional athletics (e.g. the Moneyball approach to baseball) and Nate Silver who went from pioneering the Sabermetric analysis of baseball to political commentator and analyst.

While I am a fan of analytically-driven approaches I also appreciate the potential pitfalls of relying on statistics.  More than once I have heard the quip,”lies, damn lies, and statistics”.  But it’s incumbent on consumers of data to be sophisticated consumers, such that they can call out the sloppy use or, worse, intentional misrepresentation of data.

In the current world we live in, one remains ignorant of statistics at their own peril.