Constructing Surveys Flashcards
Why is representative sampling important in surveys?
In survey research representative sampling is essential to permit generalization from a sample to a population.
How can you do questionnaire pretesting?
Questionnaire pretesting identifies questions that respondents have difficulty understanding or interpret differently than the researcher intended.
Based on an interview (debriefing) with a pilot group.
- Conventional pretesting also detects questions that respondents have difficulty understanding or interpret differently, but less reliably.
Behavior coding: Questions that elicit frequent deviations are presumed to require modification.
- Behavior coding is quite consistent in detecting apparent respondent difficulties and interviewer problems.
Cognitive pretesting, which involves asking respondents to think aloud while answering questions.
- tend to exhibit low reliability across trials and to detect respondent difficulties almost exclusively.
Define Open versus Closed Questions
closed-ended questions require people to choose among a set of provided response alternatives.
In open-ended questions, respondents answered in their own words
what is social desirability bias?
overreporting of admirable attitudes and behaviors and underreporting those that are not socially respected.
what is meant by Optimizing?
A great deal of cognitive work is required to generate an optimal answer to even a single question.
To the extent that a person is encouraged to perform the necessary cognitive tasks in a thorough and unbiased manner, a person may be said to be optimizing.
what is meant by satisficing?
Satisficing is rather than expending the effort to generate optimal answers, respondents may compromise their standards and expend less energy.
o respondents may be less thorough in comprehension, retrieval, judgment, and response selection -> Weak satisficing.
o A more dramatic approach is to skip the retrieval and judgment steps altogether -> strong satisficing.
What CONDITIONS FOSTER SATISFICING?
Satisficing is more likely to occur:
o (a) the greater the task difficulty,
o (b) the lower the respondent’s ability,
o (c) the lower the respondent’s motivation to optimize.
What is response order effects?
response order effects refer to the effect connected with the order in which response alternatives are presented.
- Written format: primacy effects, respondents want to answer as fast as possible
- Interviewer administered format: Cannot predict whether primay or recency effects.
What is the perceptual contrast effect?
Perceptual contrast effects: it may cause a moderately plausible alternative to seem less plausible if considered after a highly plausible one or more plausible if considered after a highly implausible one.
What is Acquiescence?
The tendency to endorse any assertion made in a question, regardless of its content, perhaps due to human tendency to appear agreeable or due to human cognitive biases
People who acquiesce are likely to manifest other forms of satisficing.
Explain the discrepancy between ratings and rankings.
a number of studies indicate that rankings yield higher-quality data than ratings.
Respondents are more likely to make mistakes when answering rating questions, failing to answer an item more often than when ranking
For rating questions, respondents who are inclined to implement strong satisficing can simply select a reasonable point on the scale and place all the objects at that point.
- called nondifferentiation.
Explain the SELECTION OF NO-OPINION RESPONSE OPTIONS
Many more respondents say they have no opinion on an issue when this option is explicitly offered than when they must volunteer it on their own.
By offering a NO option, respondents would be discouraged from offering meaningless opinions.
Although NO responses sometimes occur because people have no information about an object, they occur more often for a variety of other reasons.
o i.e., because people feel ambivalent about the issue or because they do not understand the meaning of a question.
o offering a NO option may discourage respondents from providing thoughtful answers -> result from satisficing.
Problems related to QUESTIONS ABOUT EVENTS AND BEHAVIORS
Lack of encoding (simply not being aware) Is the biggest obstacle
For events that respondents do encode, two major types of errors affect self-reports.
o Omission result when individual events are forgotten because of dating errors, because similar events become conflated in a “generic memory”, or because the wording of a question leads the respondent to search some areas of memory while neglecting others.
o Intrusions result when events are telescoped forward in time or when memories are altered by scripts, schemata or embellishments from retellings over time.
Problems related to QUESTIONS ABOUT SUBJECTIVE PHENOMENA
Subjective phenomena can be related to problems such as, lack of motivation to answer carefully, and response sets such as acquiescence.
What is meant by unipolar/bipolar rating questions?
Rating questions can be structured as either bipolar (ex. Extremely boring to extremely interesting) or unipolar (ex. Not at all interesting to extremely interesting).
The bipolar scale has a midpoint at which there is a transition. This can be perceived as either indifference or ambivalence.
Unipolar items make fewer assumptions, but they risk irritating respondents who see questions that present negative and positive dimensions separately as repetitive or inappropriate.
