Reader Question – How to use different surveys when their results are vastly different ?


SUMMARY

  • “If a job/position is core to an industry- or job family-specific salary survey, you might want to weigh that survey data more than the general market survey.”
  • Are you using multiple salary surveys ? In this episode, learn to think about which one to prioritise to establish the market price for a job.
  • Never forget to also look at salary survey data on a job-by-job basis to decide on the final weight of each data set when doing a market pricing.
  • Watch the video to get the full training, or read the transcript below.

Related posts

How many salary surveys should I participate to ?

The safeguards in salary benchmarking


INSPIRATIONAL QUOTES

Keep track of your reasoning when you change the standard weigh of surveys for a specific role
In general, put more weight on industry survey data than general market data

FULL TRANSCRIPT

Hello, and welcome to this episode of Compensation Insider. Today, I would like to talk about a question that I received a few years ago from one of my readers.

It’s about how to use different surveys that have very different results in the same country. The reader wrote to me :

“I’m doing a compensation survey for the entire organization, and I have two trustworthy data sources that vary greatly on a few technology positions. In order to calculate a number to use to compare and combine salaries to the market, would an average of the two data points be appropriate? I’m having a hard time with a few things about that. When I look at the variance between the two points from the two different surveys, I know from research recruiting efforts that the technology source is considerably more accurate but it’s not the source that we will be using for the rest of the company. There’s a 75% variance between the source that we’re using for the whole company versus the source that seems more accurate for our technology jobs. So I almost feel as if I should weigh the technology source more heavily but I’m asking for other opinions before I do.”

First, decide your general approach to survey weighting

If you participate in two different surveys you should decide in advance, and generally speaking, which survey would be the main source of information for which kind of jobs.

Usually, if you participate in a survey which is industry specific, or job specific, or job family specific, use that data as a primary source of information compared to a more general market survey.

So in the case of technology jobs, use the technology industry survey that might be more accurate, or if you have a marketing survey focused just on marketing jobs, use this for marketing roles. Why ? Because they will have positions that fit really well with the current trends for roles that exist in that industry, or that kind of job family.

Use the general market survey for all the positions where you are recruiting from pretty much anywhere : all the support functions for example. For instance, for an HR position, it’s better sometimes, but that’s not life and death ,if the person doesn’t have experience in a specific industry

If a job has data coming from both sources of information, you need to decide if you’re going to use only one of the surveys, or if you’re going to use both surveys and put a weight to prioritize. If the job is a position which is coming from an industry specific, or a job family specific survey, you might want to weigh that survey more if you decide to use both.

Then make an informed decision for each job one-by-one

But never forget to also look at the data on a job by job basis.

For each job the salary survey provider should tell you the number of companies that submitted data and the number of incumbents (the number of people whose salary information have been submitted). You should also have the list of companies which participated in the survey.

Let’s imagine you have a technology job, and you have data both from the general market and the technology survey on that position. Your initial way of thinking would be : “The technology survey would be a more relevant source of information so for those jobs, I’ll take the technology survey as my main source of information. Maybe accounting for 75% of the total, or I give it twice as much weight as the general market survey”, depending how you want to weigh it.

But maybe for that specific position in the technology survey, you only have 5 companies that submitted 10 incumbents, while in the general market survey for that specific job, you ended up having, for example, 55 incumbents from 22 different companies. In that case, for that specific position, I would do the market pricing by prioritizing more the data that is coming from the general survey as an exception to my general rule that I would normally prioritize the industry survey.

Now, based on other information that you have, like ease or difficulty to recruit, how different is the data compared to your internal pay range, do you have high turnover or not, etc, if one of the two surveys seems completely off, it might be better to not use it at all, rather than keep those numbers that might distort the market pricing for that position.

It’s really important that even though you have a set of internal guiding points on how you’re going to weigh, you do that on a job by job basis to establish the market values, especially if those positions are core for you in terms of your business success. Obviously if you’re looking at positions that are not so central to the organization because they’re not your core strategic type of positions, then probably you’re just as good not considering the industry data and just deprioritizing it.

Just try to think in advance what would be your theoretical approach to be consistent across all positions. If you’re going to do the market pricing and you get a question from management about why you gave more weight to that survey or that survey, you will have the reason why, and you’re credible with your decision.

Finally, validate your decision for unique/core/important roles

In the past when there was some doubt or some job families that were really important for the business, once I had done the initial market pricing, I would come back to the business and say “Look, this is how we’re looking at it. This is my methodology, this is how we come to those numbers, what do you think?”

Obviously, I was not giving an opportunity for managers to make me change my whole approach. But their feedback was important sometimes to help me tweak or understand why there was a difference. They might be able to help me guide in my thinking. The software we used had a good way for us to track those kind of decision factors, which would allow us to remember why we had made that kind of decision, even if revisiting the case one year later. That was really useful.

If you don’t have that in your software, just take some notes in a document and put it on your share drive, and make sure that it’s available for next year when you’re going to do the analysis again. This is going the be helpful whether the situation has changed completely and you want to change the way you’re doing, or you have a reinforcement of the decision that you took, so this is going to be helpful as well.

I hope this was useful for you and if you have seen some other situations and you want to share, please don’t hesitate, share in the comments. I look forward to seeing you next week in the upcoming episode of Compensation Insider. Thank you very much.

Print Friendly, PDF & Email

Speak Your Mind

*