Chapter 9: Non-experimental_survey Flashcards
(39 cards)
1
Q
For what purposes are surveys used?
A
- to determine how people feel about a particular issue, such as gun control or the performance of the president of the United States
- attempt to find out the effect of some event on people’s behavior.
- surveys provide an opportunity to examine correlations among the participants’ responses and to look for possible patterns of cause and effect, such as showing that bullying in childhood is related to internalizing problems later in life
2
Q
characteristics of survey
A
- First, surveys generally involve sampling.
- Second, surveys involve the self-reporting of behaviours, feelings, thoughts, and preferences, either in oral or written form.
- Third, all respondents answer the same set of predetermined questions to allow for comparison of responses.
3
Q
a sample
A
(i.e., a subset of individuals)
4
Q
population
A
all individuals of interest to the researcher
5
Q
sampling frame
A
a specific list of the individuals in the population
6
Q
Each individual who falls within the sampling frame is called
A
an element
7
Q
andom sample:
A
a sample in which every member of the population has an equal and independent chance of being selected
8
Q
haphazard sample:
A
population subgroup for whose selection the researcher uses hit-or-miss methods
9
Q
purposive sample:
A
a nonrandom sample that is chosen for some characteristic that it possesses
10
Q
convenience sample:
A
a nonrandom sample that is chosen for practical reasons
11
Q
A sample is representative of the population when the sample has the same — as the population
A
distribution of characteristics
12
Q
When some segments of the population are over-represented in the sample, — results
A
selection bias
13
Q
Face-to-Face survey advantages
A
- interviewers can establish rapport with the people being interviewed.
- Interviewers can direct the attention of the respondents to the material and motivate them to answer the questions carefully
- guarantee the order in which questions are administered, thus making sure that people answer the survey in the order intended.
- interviewers may be able to notice when respondents seem to misunderstand a question and explain its meaning and can probe for more complete answers when a respondent gives a brief answer or one that does not respond to the question.
14
Q
Face to face disadvantage
A
- The presence of the interviewer creates a social situation that may result in biased responses
- Respondents say what they think interviewers want to hear
- much more expensive
- safety of the interviewers
15
Q
Written response (mail)
Advantage
A
- very efficient use of time and money and can have a very high response rate if attendance by group members is high
- Low cost
- respondents can complete the questionnaire at their leisure, and they have greater anonymity in their responses, reducing interviewer bias
16
Q
Mail Survey Disadvantage
A
- The main problem with written questionnaires is response rate
- often less than 50%
- low response rates may invalidate the results because of differences between individuals who respond and those who don’t
- There is no possibility of clarifying questions that might be misunderstood. (illiterate or have vision problems)
- Illiterate participants are frequently embarrassed to admit their problem and sometimes respond to questions they do not understand, compromising the reliability and validity of the data.
- impossible to determine how seriously the respondent took the survey
- i.e. given to a child in the family to fill out
17
Q
internet survey
Advantage
A
- social desirability may be reduced
- all the questions were asked in order, and none were skipped.
- The computer can check for invalid responses and prompt the interviewer to recheck implausible answers, such as the presence of 20 children in the home
- control the sequencing and branching of questions so that, for example, people who do not drive will not need to be asked
- 24 hours a day, making a high level of participation possible without the costs of photocopying or postage
- some indications that Internet surveys have higher predictive validity than data collected by telephone without sacrificing measurement reliability
18
Q
Internet survey Disadvantage
A
- Illiterate or uncooperative participants will pro- vide meaningless data
- True measures of response rates are difficult
- typically rather low
- might not be honest about their ages or genders
- random sample of respondents may also be difficult to obtain because people participating in a Web-based survey are self-selected
- security of responding is vital. Further, storage of survey information obtained electronically must be ensured
19
Q
telephone survey
Advantage
A
- low cost
- involve less effort than some other methods of administration
- rapidly, without having to wait for interviewers to travel to many locations or for respondents to mail back their completed surveys
- possibility of using a computer-assisted interview/the interviewer reads the questions from a computer screen
- can be conducted from a central location where the interviewers can be supervised
- no reliable evidence that the data collected from cell phone–based studies are poorer in quality than landline- based surveys
20
Q
telephone survey Disadvantage
A
- Some states prohibit using auto-dialers
- since a person with a cell phone can be physically located almost anywhere, sampling appropriately from within a spe- cific location, such as everyone who may live within a particular town, is extremely difficult.
- introduce the possibility of interviewer bias
- must also be relatively short to get participants to finish the survey
- harder to establish rapport or to judge the degree of seriousness
21
Q
What are the relative advantages of using open-ended questions in a survey?
A
- Using open-ended questions makes it more likely that the questionnaire will discover something not anticipated by its designers
- more useful for smaller and preliminary studies.
- Coding a small number of open-ended surveys may be manageable, whereas hundreds would not
- trying out a preliminary version of a survey with open-ended questions can determine the range of likely answers, permitting you to standardize the alternatives into a closed-ended format
22
Q
What are the relative advantages and disadvantages of using closed-ended questions in a survey?
A
easier to code and analyze, and there are fewer off-the-wall responses
- alternatives are presented to the respondents, so they do not have to think as hard
- do not need to be as articulate to formulate their answers
23
Q
What are the relative disadvantages of using open-ended questions in a survey?
A
- harder to code, however, because the answers are in narrative form makes data analysis a messy job and makes it likely that you will have to break a cardinal rule of research by not deciding in advance how you are going to analyze your data
- issues being studied may be too complex to reduce to a small set of alternatives, or the respondent may not agree with any of them, resulting in simplistic answers
24
Q
What are the relative disadvantages of using closed-ended questions in a survey?
A
- issues being studied may be too complex to reduce to a small set of alternatives, or the respondent may not agree with any of them, resulting in simplistic answers
- tend to put words into the mouths of respondents, suggesting alternatives that respondents might never come up with them- selves
- errors can creep into the closed-ended questionnaire
25
criteria for a useful questionnaire item
* Must be reliable and valid
* Because it is reactive, i.e. They know they are being examined, they may not tell the truth - social desirability effect
* May have hidden agendas so have to design accordingly
26
Six steps in developing questionnaire and consider type of questions, the clarity and conciseness of questions, and the ordering of questions
* 1) decide what information is required;
* (2) decide what type of questionnaire will be used;
* (3) draft and then revise the questionnaire;
* (4) pretest the questionnaire;
* (5) edit the questionnaire; and
* (6) specify the procedures for its use.
* be written in language familiar to all respondents.
* be clear and specific. Define the terms and the context clearly. Make alternatives clear, exhaustive, and mutually exclusive in the case of closed-ended questions.
* avoid any leading, loaded, or double-barreled questions, which would result in biases in responses (e.g., socially desirable responses).
* be as concise as possible.
* all conditional information be presented prior to the key idea
27
Describe how the ordering of questions in a survey may influence the quality of data
* Mail
* ask first those questions that are of greatest interest to the respondents so as to capture their attention, and to leave the questions about demographic data at the end
* Personal or telephone
* establish rapport with the respondent. Such interviews often begin with questions that are easy to answer, such as demographic questions
* The generally accepted method is to begin with the most general question and then move on to more specific questions within a general topic. Often, branching items are used to direct respondents to skip inappropriate questions. The use of branching items reduces the demand on the respondent’s time.
28
the way that a question is asked is very important and can easily become a source of ---
**bias**
Do you believe in killing unborn babies?
The second one’s newsletter asks, Should women be forced to bear unwanted children?
carefully constructing the survey questions, such that the frame of evaluation is clear, helps to avoid bias
29
acquiescence:
the tendency to agree with a statement on a questionnaire, regardless of its content
30
qualities of aqcuiesence
Agree/Disagree or True/False because they are easy to administer, these formats are highly susceptible to bias caused by acquiescence
Participants exhibiting acquiescence will have a tendency to agree to any statement on the inventory, regardless of its con- tent. The effect of this bias is that participants will agree with both a state- ment and its opposite, so, for example, will respond yes to both I like fish and I don’t like fish.
It is estimated that the effect accounts for approximately 10% of responses
31
response rate:
in survey research, the percentage of individuals in the sample who return the completed survey
32
increasing return rate for mail survey =
* The response rate for mail surveys is highest when the questionnaire has a “personal touch,”
* the effort required from the respondent is minimal,
* the topic of the survey is of interest to the respondent,
* and the respondent identifies with the sponsoring organization or the researcher conducting the survey
* Researchers can increase the return rate by administering the survey in groups if attendance by group members is high
33
two limitations of survey research
* **Social desirability effect**
* **Hard to ascertain patterns with open ended questions** though these are the ones that would be more nuanced
34
Briefly describe the ethical issues in conducting survey research. What steps should researchers take to ensure that ethical principles are followed?
* anonymity and confidentiality - take measures to ensure both
* Informed consent is another important ethical guideline to be followed. A researcher must let respondents know that their participation is voluntary, that they can choose not to answer any of the questions in the survey, and that they can terminate their participation at any time, without any penalty
35
mutually exclusive questions:
categories defined so that membership in one rules out membership in another
36
exhaustive questions:
categories defined so that all possible cases will fall into one of them
37
verification key:
a collection of items on a questionnaire designed to detect dishonest answers
38
branching items:
a set of questions that enable the respondent to move through a survey in different ways, depending upon the responses
39
