When creating a survey, it can be easy to focus too much on the content of the survey questions themselves and overlook the response scale that goes with the survey items. Sometimes the response scale is just an afterthought, with only a numeric scale and no further instructions. For example, “On a scale of 1-5, rate the following items.” This can confuse the survey respondents. Providing a clear response scale for your survey items will lead to more effective feedback for the feedback recipient. A simple response scale allows all respondents to be on the same page as they provide their answers. Here are some tips for creating an effective response scale for your survey.
Should Relate to Observable Characteristics
You want to ask the respondents to give feedback on something they can observe about the feedback recipient. For example, you could ask about frequency, “How often does this person exhibit this behavior?” A 5-point frequency scale might be:
- Almost Never
- Rarely
- Sometimes
- Frequently
- Almost Always
Another response scale could ask about agreement, “How strongly do you agree or disagree that this person exhibits this behavior?” Possible responses could be:
- Strongly Disagree
- Disagree
- Neutral
- Agree
- Strongly Agree
Should Be One Continuous Scale
When you create your response scale, it can be tempting to add clarifying words and phrases to each scale item. However, that can sometimes result in a response scale with multiple different scales. This extra language can make the results more confusing when analyzed. Here is an example that might create confusion.
- Weak: The person hardly ever demonstrates this behavior—there is a need for serious further improvement.
- Improvement Needed: The person demonstrates this behavior to a limited extent—extensive further improvement is required.
- Competent: The person demonstrates this behavior to a moderate extent—further improvement is required.
- Strong: The person demonstrates this behavior to a sufficient extent—some further improvement is required.
- Exemplary: The person demonstrates this behavior to a great extent—no further improvement is required.
There are really three different scales in the response items above. The first one or two words at the beginning of each scale item ask the respondent to determine how well the person does the behavior. The second scale asks how often the person demonstrates the behavior—from “hardly at all” to “to a great extent.” And the final part of each item asks the respondent how much further improvement the behavior needs.
On their own, none of these three response scales are inherently wrong or unusable. The problem is that the respondent can only indicate one response for each survey item. So, it is difficult to know if any given respondent is indicating how well the person does the behavior, to what extent the behavior is demonstrated, or how much further improvement is required.
How Many Points?
You might wonder how many points to put into the response scale. Surveys use anywhere from 3 to 11 points in the response scale for the items.
Here are some things to keep in mind:
An odd number of points—like 3, 5, or 7 points—allows the respondent to indicate a “neutral” middle answer. Choosing an even number, like 4 or 6 points, means that the respondent will have to decide whether to provide a “generally negative” or a “generally positive” response.
An even number of points in the response scale can lead to an overall positive bias in the feedback. Respondents may feel unsure about how to rate a specific item. They may feel it is socially undesirable to provide a negative rating. Consider a 4-point response scale. Respondents may choose 3 as a “neutral” answer, even though this is in the upper half of the scale. People are more likely to select the positive side if they are neutral or uncertain but forced to make a selection. The attempt to get what seems like more useful or accurate responses can actually result in an artificially inflated positive score.
This doesn’t mean that an even-numbered scale is never a good option, but in most cases, we recommend using an odd number of points in the response scale to help provide a truly “neutral” middle ground for respondents.
As part of any response scale, we also recommend including the option “not applicable” or “don’t know.” The combined feedback would not include this option. This option gives the respondent an “out” for a survey item where they may not be able to provide useful feedback.
Summary
The most effective response scales in a survey are simple. This makes it easy for the feedback provider to assess each survey item and provide wide-ranging feedback. We most often see a 5-point scale asking, “How frequently does this person exhibit this behavior?” or “How strongly do you agree that this person exhibits this behavior?” A response scale like this allows all feedback providers to be on the same page as they give their feedback.