L2.2 item Types

Question 1

What are close-ended questions?

Answer 1

• Close-ended questions ask respondents to choose from a fixed set of response alternatives.
• E.g. MCQs, yes/no and questions with a numerical rating scale

Question 2

What are open-ended questions?

Answer 2

• Open-ended questions do not provide respondents with any response alternatives.
• They are given space to respond in their own words.
• This may be long-answer
• Or short answer

Question 3

What are rating scales

Answer 3

• The most widely used response format
• The most common rating scale is the Likert scale
• Respondents provide ratings indicating how strongly they feel positively or negatively on an issue.
• Likert scales use words to ‘anchor’ the numerical ratings.
• Most surveys use between 5 and 11 points.
• Typically, respondents are not capable of making the distinctions between adjacent points in a rating scale with as many as 11 points.

Question 4

Advantages of Likert Scales

Answer 4

• Efficient to create (i.e, basic ones)
• Most respondents have experience with Likert scales.
• Using the same response alternatives for numerous items allows for the collection of a large amount of information in a relatively short period of time.
• Likert scales yield numerical scores which can be analysed statistically.

Question 5

Mid-Points?

Answer 5

• It is debatable whether you should or shouldn’t use alternatives such as ‘Neither Agree/Nor Disagree’.
• In my opinion, it is better to force people to make a decision one way or another
• Some people have personality types that lend themselves to responding ‘Neither Agree/Nor Disagree’
• They are not the type to express an opinion (even though they may have one).
• They may not want to talk about it too much
• Sometimes it is unavoidable to use a response such as ‘not applicable’, but this is not a midpoint.

Question 6

Close-ended questions: Advantages

Answer 6

• Fast and easy for respondents
• Restrict range of responses -helps keep people on the topic of the question
• Can remind people of things they otherwise would have forgotten
• E.g. Questions such as “What kind of social activities did you participate in at work this year?”
• some people might forget that they attended a morning tea for someone’s birthday.
• If one of the close-ended alternatives includes ‘birthday morning tea’, they’ll recall that they attended one.

Question 7

Close-ended questions: Disadvantages

Answer 7

• Participants are restricted to responses that may not accurately reflect their feelings or attitudes about the topic.
• May force people to express an opinion when they don’t actually have one on the topic (i.e., if none of the alternatives is ‘Not applicable’ or ‘No opinion’.)
• In some contexts (job satisfaction surveys), closed-ended questions may cause frustration in the respondents if they are made to feel that they were not allowed to express their true feelings.

Question 7

Open-ended: Advantages

Answer 7

• Can convey information that can not realistically be captured in a fixed-answer format.
• E.g. “What are your suggestions for improvement?”
• For sensitive topics (e.g. Drug use), it can more useful for people to respond in a less constrained way than a fixed-answer format.
• Also, in practice, when presenting reports to a client, it can be useful to compliment a report with some qualitative statements derived from the open-ended questions in the survey.
• Allow people to express their precise feelings/attitudes
• Provide more in-depth answers than close-ended questions
• Can give respondents an opportunity to “blow off some steam”.
• Gives people a sense that their opinion matters.
• Many managers like to read the open-ended question answers.

Question 8

Open-ended: Disadvantages

Answer 8

-Take more time for the respondent to complete
-People may get tired/bored by the end of the survey and answer the last questions perfunctorily.
-Some people will be uninterested in the survey and will give poor open-ended responses.
-Very time-consuming to analyse.
-Are not quantitive (they don’t measure intensity or satisfaction directly).
-Some people aren’t good at providing long responses.
-Often, people “unload” rather than provide responses to the question.
-E.g. In the SPOT forms, there is the following open-ended question:
-‘Please list any suggestions that will help improve teaching and your learning in this unit.”
-“Where does one START!”
-“Come out of the closer!”
“…evidently sexually frustrated.”

Question 9

Open-ended Alternative

Answer 9

• An alternative to truly open-ended questions would be ask people to list the top 3 reasons for something.
• E.g., ask people to list the top 3 reasons why people quit the company.

Question 10

What is a positively keyed item?

Answer 10

• A positively keyed item entails that the person is affirming the attribute of interest.
• E.g., Openness to Experience
• Q: I get a feeling of excitement when I try new things
• A: Strongly disagree to strongly agree

Question 11

What is a negatively keyed item?

Answer 11

• A negatively keyed item entails that the person is denying the attribute of interest.
• E.g., Openness to Experience
• Q: I prefer to stick with what I like, rather than risk disappointment by trying something new.
• A: Strongly disagree to strongly agree

Question 12

What is a double barrelled item?

Answer 12

• A double-barrelled item asks 2 or more questions in the same item which may be expected to be answered differently by at least some respondents.
• Generally include the word ‘and’ or ‘or’ in an item, which should be avoided.
• Keep the items simple and focused on one idea.
• E.g. ‘I found the training program pleasant and useful’.
• Some people may have found the program pleasant, but not useful.
• How should they respond to such an item?
• You’d be surprised how often double barrelled items exist in published tests.

Question 13

Use simple language

Answer 13

• In most cases, you should use language that requires an 8th grade level or lower.
• Even if it is a questionnaire for adults.
• Avoid slang terms, colloquiums and jargon.
• E.g. “I feel like I have the weight of the world on my shoulders.” May not be an expression in a different culture or language.
• “I would be elated if my gf became preggers.”

Question 14

Items should be short

Answer 14

• All other things equal, an item should be short.
• Many people don’t have long attention spans, so anything over 15 words is a risk.
• Generally, any item that has more than 1 comma is too long or complicated.