We get to work with a lot of great people who want to provide a way for their clients or employees to receive feedback. During this process, we often help develop, revise, or refine survey items. We’d like to share 10 tips about common issues we have seen with creating survey items over the years. You can also download a checklist version.
Anonymity
- If you’re going to ask the participants to provide demographic information, be sure to avoid demographic questions that are too specific and could reveal their identity.
Wording
- Avoid acronyms, jargon, and abbreviations—including corporate-specific or HR-specific terms. If your audience doesn’t understand the question, they can’t respond accurately. We find it is best to use plain language.
- Avoid ambiguous questions and or vague words that have multiple meanings.
- Ensure each item addresses only one concept or idea. Items that contain multiple thoughts are called, “double-barreled questions”. Instead of “Does he perform well on the job and promptly meet deadlines?”, break it down into two separate questions: “Does he perform well on the job?” and “Does he promptly meet deadlines?”.
Negatives
- Questions should not contain double negatives. Instead of “His comments in meetings aren’t unnecessary” use “His comments in meetings contribute to the agenda”.
- Be careful with “negative” questions that would be reverse-scored. For example, “Is primarily interested in his/her own welfare.” If this question were asked on a 5-point agreement scale, then a desirable rating on this item would be “Disagree” or “Strongly Disagree”. This puts an extra burden on the reader to think carefully about the meaning of the question and understand that the “best” answer might be different from the rest of the questions in the survey. When there are just a few of these questions in the survey, they can trip up a respondent who isn’t paying careful attention.
Scale
- Keep your response scale in mind as you compose the questions. For example, if you’re response scale is a frequency scale (Never, Rarely, Sometimes, Very Often, and Always), you’ll want to avoid an item that mentions frequency in the item. “Is always quick to follow-up on concerns.” This is potentially confusing to a reader—are you asking the respondent to tell you that the person is very often always quick to follow-up? Or never always quick to follow-up? Instead, drop the frequency reference out of the item itself and use something like, “Follows up on concerns.” This makes it much clearer as the respondent considers how frequently the person follows up on concerns.
- Avoid questions with an unbalanced response scale. For example, rather than the scale “Disagree; Agree; Strongly Agree”, use “Strongly Disagree; Disagree; Neither Disagree nor Agree; Agree; Strongly Agree”. The second option has a mid-point with equivalent options on each side.
Length
- Get to the point and stay with it. Write questions that are relevant to your goal. It is important to maximize the usefulness of your audience’s time. Your audience should be able to clearly see an association between the purpose of the survey and the questions asked.
- Keep the survey short. Try to keep it between 25-40 questions, taking no longer than 20 minutes to complete. There may be times not to follow this tip; however, the most common reason isn’t sufficient, which is: “As long as I’ve got their attention, I might as well ask these other questions I’m interested in as well.”
Keeping these points in mind as you compose your survey items will help ensure the survey is easy to understand and will provide useful results.
Related article:
What to Look for in a 360