Staff surveys: lessons from Market Research

Staff surveys are used a lot but they are not always useful. Running the same annual survey gives us year-on-year comparison data, but the questions can lose meaning or focus as the organisation changes. After all the work of designing, administering and analysing a new survey, we often end up with information that either doesn’t help (‘That’s not we really wanted’) or seems trivial (‘We already know that’). Part of the problem is that we don’t know the difference between what Market Researchers call Qualitative and Quantitative research.

Quantitative research (‘quant’) collects data that can be simply coded numerically: typically, ‘Yes’, ‘No’ and multiple choice responses to straightforward questions. Quant needs large samples of respondents (hundreds) because large samples enable statistical analysis that yields reliable and valid results. The trouble is, quant can be time-consuming and resource-heavy, and if the wrong questions are asked then all that is wasted. And since quant is often all we do, the process of surveying staff gets a bad reputation.

Qualitative research (‘qual’) collects data that is not easily quantifiable or numerically coded. It usually concerns people’s ideas and feelings, which can be complex and not easily summarised. It’s often collected in in-depth interviews or focus groups: the answers to open questions such as ‘Why?’ and ‘How?’ Qual is done to ‘scope’ or explore a range of opinions. An objective, experienced and emotionally intelligent qual researcher can get a vast amount and variety of information out of a few meetings. They can put questions in their organisational context. They can also ensure that respondents reflect upon questions before answering: they don’t get ‘gut feel’ or ‘knee jerk’ responses. But the number of people (staff and key stakeholders) involved is nearly always too small for their answers to yield statistically meaningful results: we can’t measure the relative popularity or importance of the ideas and opinions collected.

So what’s the answer? Do both!

Begin with qual. Use an experienced and objective outsider to scope the area you want to survey. Gather the full range and variety of issues. Produce an interim report. Then – if your sample is large enough to enable meaningful statistical analysis – write your quant questions, distribute your quant questionnaire (email, paper and online), collect and the interpret responses and write your final report. This way you’ll focus your staff survey on what’s really important, and the expensive part – the quant – will not be money wasted.

Quant AND qual. That’s how professional Market Researchers do it.