Combining two different surveys into one summary is not easily done and is generally inadvisable unless you have all the underlying observations.
Questions raised in other online communities by people not well versed in compensation practices often trigger concepts that should be shared with a wider audience. A more extensive response to that particular issue is being posted here, not because we need to know it, but because it is one FAQ whose answer is not easily found anywhere. Hopefully, this is the place.
Question: If I get one market survey from Hay Group and another survey from Mercer, can I weight these 2 market data reports into 1 market data report?
I asked some Sr. C&B professionals, but they told me, "no." Is that right?
Long Answer: They are correct that it is technically improper to co-mingle and compress two surveys into one because you don't possess the exact underlying observations of each survey. Thus, you cannot "weigh" them statistically because you don't know their relative values and frequencies in order to establish valid weights that give more influence to one dataset than another.
The best you can do is to offer some subjective assumptions which always turn out to be wrong, anyway.
For example, you have no way of knowing if both Hay and Mercer surveyed exactly the same firms at the same time, which means their data are identical and should not be weighted; or Hay might survey your closest peer competitors while the Mercer survey might focus on outfits unlike you whose pay practices are totally irrelevant, in which case you would not want to use any of the Mercer data... perhaps. Or the Mercer survey could cover twice as many of your rivals while the Hay survey might be smaller and more current but based on Hay evaluation points rather than benchmark title-matching; so you would have a handful of contributing variables to juggle in your weighting process. Without access to all the detailed source information held by the surveyors, it is impossible to mash their different surveys into one comprehensive single summary that would meet acceptable statistical reliabilty standards.
When you lack all the original observation responses, any effort to combine separate data sources involves guesswork and subjective assumptions. Mash-ups are further complicated by the DOJ/FTC Safe Harbor provision affecting U.S. surveys. Those American agencies prohibit law-abiding objective third-party pay surveyors from sharing those underlying details. You are simply S.O.L.
If your boss or client insists that you mix two or more surveys into a single result, you can tell them “that’s bad science and poor math, but I can offer a guesstimate.” Or you can distract them by asking “why would you want to lose the advantage of having multiple views?” That response reflects the Federal Acquision Regulations that recommend as Best Practice guidelines that any important job value should be established by reference to three independent survey sources. Note that they recommend that you cite three or more measures without explaining exactly how to arrive at an answer they will ACCEPT. Sneaky, these Feds… always leaving an escape route for capracious rulings.
When required to combine multiple surveys despite those warnings, I suggest you just say, "The consensus seems to be about (insert value)," or "The answers cluster around (insert value)." Any other answer will expose you to all the irrefutable criticisms offered here.
How else have you handled this frequently-encountered issue?
E. James (Jim) Brennan is Senior Associate of ERI Economic Research Institute, the premier publisher of interactive pay and living-cost surveys. Semi-retired after over 40 years in HR corporate and consulting roles throughout the U.S. and Canada, he’s pretty much been there done that (articles, books, speeches, seminars, radio/TV, advisory posts, in-trial expert witness stuff, etc.), and will express his opinion on almost anything.
Creative Commons image "blender blender blender blender" by miss karen