Numbers That Lie

Hard numbers can turn out to be really soft and squishy upon closer examination.  Perhaps that is why predictive foresight is rarely fully accurate, even when based on the Best Available Data.  (Note the nifty acronym, proving that even an optimal decision is still doomed to be a BAD decision.)

Reliance on data¹ does increase the certainty people feel when making their mistakes.  Numbers are therefore fondly embraced by most compensation people, because numeric data seem to confirm the accuracy of measurements.  But those very numbers can cause us to stumble because they promise a degree of precision that can be quite misleading, if not downright false. 

Last year, the initial Apple effect greatly skewed exec comp survey results (see The Apple Effect note at the bottom here).  Some frowned at the $376,000,000 in stock options granted to the new Apple CEO Tim Cook (which temporarily ballooned in value to over $600 million shortly thereafter) but little attention was given to his subsequent 99% pay cut this year… perhaps because it’s all RELATIVE.  Funny, how a relative increase gets headline attention but a relative decrease is ignored.  Surveys that did not make allowances for massive swings in isolated observations like that would leave their followers spinning in confusion.

Maybe it all depends on whether you want to find what is normal (the median), discover the most popular (the mode) or simply compute the average.  Those arcane terms are really important in compensation work, because each has a different precise meaning. 

When Bill Gates enters a Starbucks, the average customer in the shop becomes a millionaire.   Averages are directly impacted by outliers, while medians are the middle figure no matter how high the high or how low the low observation.  Conventional society tends to favor AVERAGES even though they are more problematic in statistical terms.  There are all sorts of fancy tricks statisticians use to manipulate the figures they process into “hard numbers.”  They include winsorizing, trimming, hot decking (where a missing value is replaced by a similar but different observation) and others .  Excuse me, but some of those gimmicks strike me as flat-out falsehoods; yet they are enshrined in statistical lore and thus may be applied without your knowing.

Likewise, anyone who finesses you with a claim that pay falls in a bell-shaped curve is yanking your chain (or pulling your thumb) in a big way. Wages are never symmetrical nor do salaries have a normal distribution.  That means some of the standard statistical techniques just don’t fit the world of total rewards where the observations are living, breathing human beings.  Whenever the science of number-crunching is invoked, the art of judgment must be added to assure its proper application.

These are just a few of the common compensation conundrums of lying numbers.  What are your favorites?

¹Mark Twain - There is something fascinating about science. One gets such wholesale returns of conjecture out of such a trifling investment of fact. 

E. James (Jim) Brennan is Senior Associate of ERI Economic Research Institute, the premier publisher of interactive pay and living-cost surveys. Semi-retired after over 40 years in HR corporate and consulting roles throughout the U.S. and Canada, he’s pretty much been there done that (articles, books, speeches, seminars, radio/TV, advisory posts, in-trial expert witness stuff, etc.), and will express his opinion on almost anything.

Creative Commons image "Bell Curve" by hardeep singh

Comments

[Jacque Vilet]

The "art" of compensation is ---- knowing what is science and what is "art". If you get my drift . . . .

[E. James (Jim) Brennan]

Good observation, Jacque. Or maybe the actual science of compensation involves knowing how much of what is presented as "science" is really just "art." Almost everyone has had some experience with artistically creative applications of scientific terms in ways that communicate "truthy" ideas that are not correct despite their impressive wrappers. Technical scientific terms can be slapped on things to imply a degree of reliability and authority that simply does not exist. That kind of “art” is often called “propaganda,” “marketing,” or “advertising.”

Gotta be careful, out there.

[Dan Walter]

Jim,
Well written and important reminder. Too often we get stuck in the "that's what the says says" mode. We need to remember that before it was data, it was the same, often messy, pay for individuals.

[Jeffrey Haynes]

Jim - excellent post. I loved the line about Bill Gates walks in and everyone becomes a millionaire(on average) - does this mean he starts handing out money ? Ha !

When talking about Comp Data, I refer to the definitions of two words. "accurate" and "precise". Sure its subtle. But if I am training a Comp Analyst on what I expect of them, how they should interpret data, and how they should represent it to managers I would much rather them work towards "accurate" (free of errors as a result of being careful and thorough) vs telling managers that the data (and its analysis) represent a "precise" fact (minutely exact and perfectly conforming).

Again - excellent post.... with permission, I may use that Bill Gates line !

[E. James (Jim) Brennan]

Thanks, Dan & Jeffrey. It is wise to always be aware of the limitations of what words really mean. Living North of Seattle, I can't claim ownership of the Gates line, which simply illustrates the way understandings slide to imply that an average is what individuals possess rather than a computed number that may not match any actual observation at all.

[Chris Dobyns]

Thought this was quite good - given that Jim and I recently exchanged observations about the issue of "predictive analysis" and the apparent change-able disdain (or endorsement) of that process by a colleague.

In my time working the D.C. area, I've become sensitized to the often-made (and rarely challenged) statement of people to offer both "facts" and "true facts". However, I've not yet figured out what distinguishes one group from the other.

[John Nichols]

Compensation management really is an art and not a science and the market is only one bit of data to take into account in setting an appropriate rate in your company.

The skill is generally acquired with experience rather than just examining spreadsheets.