That's what a client asked me the other day. The SVP of Human Resources was mulling over my market pricing report when he said, "These summary averages tell me how my pay practices compare against my competition, right? And with these individual job comparisons, they tell me what the market is paying against my jobs, right?" Then a pause, followed by, "And you're sure about these figures?"
In other words, the client wanted a guarantee that what he was looking at truly reflected the competitive marketplace for his organization. So that he could take my word to the bank. And fall on his sword if necessary. Because he has the answers.
Ahhhh, but it doesn't work that way. There are no ironclad guarantees, no hand writing in the sky, no finger pointing to an "X." Practitioners who analyze compensation surveys can only report on what the sources show. It's how they, and their client interpret and ultimately utilize those reported figures that will tell the tale for the organization.
An Imperfect World
Let's consider the possible variables that prohibit you from making a pinky swear regarding the figures you've just shown to management.
• The client may be interested in what companies X, Y and Z are paying, but unless they're prepared to sponsor a custom survey what they'll get instead is a broader view that displays what many organizations are paying - some of which they may not be interested in.
• Or perhaps management is only interested in "like" industries, or only like revenue size or like geography (or, they want all elements at the same time). However, matching data slices may not be available. You may have had to pick and choose your points of comparative analysis.
• Surveys never capture what everyone is paying, but only a representative sampling.
• An oft-forgotten aspect of surveys is that the input completion process (that dreaded questionnaire) is typically presumed by those on the receiving end as a virtuous document well scrubbed of any errors. Probably not. A little garbage always slips in.
• Clients tend to think about what the market looks like today, while even the latest commercial surveys are only reporting data that was captured six months ago. And a typical aging process uses a flat growth percentage figure for all jobs; so some could be overestimated, or the opposite of what things look like today.
• And if the client's organization has caught the infection called title inflation, the job matching exercise might lose a bit of precision.
So the figure you're looking at may not be quite as bright and shiny as you'd like. You'll still have to work with it, make sense of it, and apply a dab of professional judgment. Use it less as an "x marks the spot" and more as a "let's move in this direction."
What To Do
All a practitioner can do is to present the best information that's available. That information should be used by the client as part of their decision-making process (competitiveness, structure change, merit increase process, etc.), not as the whole and sum. Survey figures should not be considered the answer, because they're not.
Unless of course your compensation strategy is simply to follow the leader, whatever everyone else is doing, or whatever the surveys say.
When They Ask For A Guarantee
When management wants some sort of guarantee that what you're showing as the marketplace in your analysis is really the market, the best tactic is to take that opportunity to educate them on how surveys work; what they can tell you and what they can't. Scrape off a bit of the mystique of precision that often surrounds how commercial surveys are marketed.
Because there are no guarantees. And you can only be as positive about those survey figures as your professional judgment allows.
Sorry.
Chuck Csizmar CCP is founder and Principal of CMC Compensation Group, providing global compensation consulting services to a wide variety of industries and non-profit organizations. He is also associated with several HR Consulting firms as a contributing consultant. Chuck is a broad based subject matter expert with a specialty in international and expatriate compensation. He lives in Central Florida (near The Mouse) and enjoys growing fruit and managing (?) a clowder of cats.
Creative Commons image,"God Complex," by angelofsweetbitter2009
So, we are talking about a general industry survey, either for a particular industry with companies in different sizes or for a region with even differing industries). The most important contributor for data accuracy is undoubtedly the job matching exercise. You might not have control as a participant, but whoever is overseeing the survey can do a lot to mitigate the anomalies. It goes without saying that this would be easier for a long running survey than one that just started a year or two ago.
In addition, most surveys would allow you to produce a custom report with a selected peer group. This would have to adhere to certain rules to avoid any litigation. Overtime, you would be able to achieve a certain level of comfort vis-à-vis the accuracy of the data.
I think, that one of the major fallacies with these surveys is the focus on actual pay rather than on pay policies. Actual pay fluctuates and more so if you are looking at a specific job with fewer occupants. I would prefer to know what is the average basic and total pay policy for a certain job or grade in addition to the actual pay. Comparing the two would provide a higher level of accuracy.
We could also minimize the inaccuracies if we were to clearly define what is included in total pay and what is not to be included.
At the end, you are right. As much as we could do individually to improve the accuracy of the data, we must always take with a grain of salt. It will never be 100% accurate as long as it is subject to human interpretation.
Posted by: HASHIM M. ALNAJJAR | 12/30/2015 at 12:36 AM
Hi Chuck!
Here's a few other issues with surveys...
* Often the least experienced compensation person is filling out the survey
* Companies dropping out or joining the survey can impact individual jobs with smaller sample sizes
* Some surveys report only actual data while others provide target and actual
* Some surveys leave out important information such as LTI data
* Practitioners may interpret LTI data as part of TDC for all when only a small percentage of participants are actually eligible or receive it
...are there other challenges we can think of?
Posted by: Leah Davis | 12/30/2015 at 08:29 AM
Chuck spoke well on a popular topic. Here's another take on survey implications: http://www.compensationcafe.com/2015/01/why-pay-surveys-rarely-agree.html. Seems like there are more ways to mislead than there are to properly enlighten, perhaps because the anxious demands for absolute certainty are impossible to satisfy. In the real world, statistical reliability involves standard error. And the findings may be irrelevant, too.
Posted by: E. James (Jim) Brennan | 12/30/2015 at 07:39 PM
Errata: a final period disables the link I posted above. Remove that period, and it works again.
Posted by: E. James (Jim) Brennan | 12/31/2015 at 02:48 PM