It may seem a little tangential, but I like this quote when I think about tips for creating survey items.
“I have a new method of poetry. All you got to do is look over your notebooks… or lay down on a couch, and think of anything that comes into your head, especially the miseries… Then arrange in lines of two, three or four words each, don’t bother about sentences, in sections of two, three or four lines each.”
– Allen Ginsberg, 1952
If you try to use Allen Ginsberg’s method of poetry to create your survey items, you’re going to be in a world of trouble when it comes to interpreting the results of your survey! Let’s look at three tips for creating survey items that will help ensure you get meaningful results. This is a follow-up to “10 Tips for Creating Survey Items.”
Discrete and Complete
First, keep this approach in mind: “Be discrete and complete”. To be discrete, you want to be sure that the topic of each survey item does not overlap with the topic of another survey item. If survey items overlap, then it will be difficult to roll up the items into a meaningful dimension score. For example, if there are five items in a dimension, but 2 of them overlap—measuring essentially the same behavior—then that behavior is getting twice as much weight in the dimension score as the other three behaviors.
In addition to being discrete, you want your survey items within a dimension to “completely” cover the entire meaning of the dimension, without leaving a gap. Of course, this can be tricky to do while still keeping the overall survey from getting too long. But making your survey items “complete” will help ensure that a rolled-up dimension score is meaningful in the end results.
Simple to Read and Understand
When composing your items, compose them as if you were asking them face-to-face. You’ll need two versions of each item: one for the participant (self-assessment) and one for the respondent (giving feedback to the participant). I like to “speak” it out loud in my head with a lead-in to each survey item. For the self-assessment items, put “You…” in front of each item. For example, “You create a vision of what the unit can become in the future.” For rating others, you can put your own name at the beginning, like “Troy creates a vision of what the unit can become in the future.”
In following this formula, be careful to start each item with a verb or adverb. Some items might need to start with “is”. For example, the item “Patient in working with others” doesn’t work with the lead-in of the participant’s name, “Troy…patient in working with others.” Instead, it needs “is” as the verb at the beginning, “Troy…is patient in working with others.”
Wording for Open-ended Items
I find this approach is especially important when considering open-ended items, since it is sometimes easy to create an open-ended question that works well in the third person (when rating someone else) but doesn’t apply very well for a self-assessment. For example, “If you were advising this person on how to become more effective as a leader, what changes would you suggest?” could be difficult to turn into a question on a self-assessment. Do you say, “If you were advising yourself on how to become more effective as a leader?” That seems like a strange way to ask the question. Instead, you could re-word the self-assessment question slightly, like “If someone were to give you advice on how to become more effective as a leader, what do you think they would say?” or even more simply, “What are some ways you could become more effective as a leader?”
If you keep these tips in mind as you compose your survey items, they will help you come up with a survey that is “discrete and complete”, that is simple to read and understand, and will provide meaningful and useful results to the participant.