Chapter 7 - Survey Research

Question 1

Self-Report Measures

Answer 1

A statement or series of answers to questions that an individual provides about his or her state, feelings, thoughts, beliefs, past behaviors, and so forth.

Self-report methods are particularly useful when measuring thoughts, feelings, or behaviors that cannot easily be directly observed.

Question 2

Self-Report Bias

Answer 2

A methodological problem that arises when researchers rely on asking people to describe their thoughts, feelings, or behaviors rather than measuring these directly and objectively.

People may not give answers that are fully correct, either because:
*they do not know the full answer
*because they seek to make a good impression.

Question 3

Response Set

Answer 3

A tendency to answer questions in a systematic
manner that is unrelated to their content.

Participant introducing measurement error

Question 4

Three types of content measured by survey questions:

Answer 4

*Facts and Demographics
*Attitudes and Beliefs
*Behaviours

Question 5

Question Wording

Answer 5

Questions must be carefully designed to maximize the likelihood of eliciting informative responses and avoiding potential problems:
*Unnecessary Complexity
*Double-Barrelled Questions
*Loaded Questions
*Negative Wording
*“Yea-Saying” and “Nay-Saying”
*Fence Sitting

Question 6

Factor Analysis

Answer 6

A broad family of mathematical procedures for reducing a set of interrelations among manifest variables to a smaller set of unobserved latent variables or factors

For examples, a number of tests of cognitive ability might be intercorrelated to enable factor analysis to reduce them to a few factors, such as executive function, processing speed, and attention

A big set of data and understand them by categorizing them to see them underlying ideas and see how they relate to eachother

Question 7

Open-Ended Questions vs Close Ended Questions

Answer 7

Open-Ended Questions: questions that allow respondents to answer in any way they wish, with no restrictions

Pros:
- Good ecological validity
- Unlimited response options!
- Good for looking at themes of data
Cons
- Unlimited response options
- How do you code and quantify the data?
- This process can be extremely time intensive

Closed-Ended Questions: questions that offer respondents a limited number of response options

Pros:
- Easy to code and quantify
- Limited response options
Cons:
- Limited response options
- Designing questions can be challenging

Question 8

Rating Scales

Answer 8

An instructment that is used to scores to people or items along some numerical dimension, such as agreement with an attitude statement or frequency of occurrence

Question 9

Likert-Scales

Answer 9

A type of direct attitude measure that consists of statements reflecting strong positive or negative evaluations of an object. Five-point scales are common and a neutral middle point may or may not be included

For example, an assessment item using a Likert scale response format might include the following statement choices: strongly disagree, disagree, neither disagree nor agree, agree, and strongly agree.

Question 10

Non-Verbal Scales

Answer 10

A type of rating scale that does not require language

These are particularly useful with populations where communication may be difficult (eg. children)

Question 11

Forced-choice Question

Answer 11

A test or survey item in which several possible responses are given and participants are asked to pick the correct response or the one that best matches their preference.

Question 12

Response Rate

Answer 12

The number of individuals who complete an interview, answer a survey, or join a research study compared to the number who were invited to participate, often expressed as a percentage

Question 13

Population vs Sample

Answer 13

Population
- A theoretically defined, complete group of things from which a sample is drawn to obtain empirical observations and to which results can be generalized

Sample
- A subset of population of interest that is selected for study which the aim of making inferences to the population

Question 14

A Sample as the Population

Answer 14

Samples serve as representatives of the broader population.

The larger and more representative your sample, the more likely it is resemble the population of interest.

Question 15

Confidence Interval (CI)

Answer 15

A range of values for a population parameter that is estimated from a sample with a preset, fixed probability (known as the confidence level) that the range will contain the true value of the parameter.

The width of the confidence interval provides information about the precision of the estimate, such that a wider interval indicates relatively low precision and a narrower interval indicates relatively high precision.

Smaller the sample, less representative of the population

Question 16

Sampling Error

Answer 16

The predictable margin of error that occurs in studies of samples of cases or observations from a larger population.

It indicates the possible variance between the true value of a parameter in the population and the estimate of that value made from the sample data.

Therefore, it is the degree to which the estimate based on a sample deviates from the true population value.

Question 17

External Validity

Answer 17

The extent to which the results of research or testing can be generalized beyond the sample that generated them

The more specialized the sample, the less likely will be that the results are highly generalized to other individuals, situations, and time periods

Question 18

Sampling Frame

Answer 18

The individuals or clusters of individuals in a population who might actually be selected for inclusion in the sample.

Question 19

Sampling Bias

Answer 19

A systematic and directional error involved in the choice of units, cases, or participants from a larger group for study

Sampling bias can threaten the internal validity of a study if there is a possibility that preexisting differences arising form the sampling process may interact with the variable of interest

Similarly, if the procedure used to choose particcipants tends to favour especially motivated individuals or people from a certain segment of society, there would be a threat to the study’s external validity (ie inferences to a larger population would not be viable)

Sampling bias is associated with a lack of random sampling and with nonrandom assignment to conditions

Question 20

Probability Sampling

Answer 20

Any process in which a sample of participants or cases is chosen from a larger group in such a way that each one has a known (or calculable) likelihood of being included.

This requires a well-defined population and an objective selection procedure, as in random sampling.

Additionally, all members of the population must have some (i.e., nonzero) chance of being selected, although this probability need not be the same for all individuals.

Question 21

Random Sampling

Answer 21

A process for selecting a sample of study participants from a larger potential group of eligible individuals, such that each person has the same fixed probability of being included in the sample and some chance procedure is used to determine who specifically is chosen.

The main value of this form of probability sampling is its positive impact on generalizability and external validity.

Question 22

Stratified Sampling

Answer 22

The process of selecting a sample from a population comprised of various subgroups (strata) in such a way that each subgroup is represented.

The selection procedure within each of these strata may be random or systematic.

In stratified random sampling, a chance process is used to select individuals

In stratified systematic sampling an objective, orderly procedure is applied to choose individuals.

Select subgroups and random assignment in the subgroups

Question 23

Cluster Sampling

Answer 23

A tiered method of obtaining units for a study.

A population is first subdivided into smaller groups or clusters (often administrative or geographical), and a random sample of these clusters is drawn.

The process is then repeated for each sampled cluster until the required level is reached.

Question 24

Non-Probability Sampling

Answer 24

Any process of choosing a subset of participants or cases from a larger population in which it is impossible to precisely determine each unit’s likelihood of being selected.

Nonprobability sampling makes it difficult to determine how well the target population is represented by the subset, thus limiting the generalizability of findings.

Question 25

Convenience Sampling

Answer 25

Any process for selecting a sample of individuals or cases that is neither random nor systematic but rather is governed by chance or ready availability.

Data obtained from convenience sampling do not generalize to the larger population; there may be significant sampling bias, and sampling error cannot be estimated.

Question 26

Purposive Sampling

Answer 26

A sampling method that focuses on very specific characteristics of the units or individuals chosen.

For example, a researcher investigating a specific type of amnesia may select for study only those individuals who have specific lesions in their brains.

Although the final subset of cases is extreme and not random, valuable information nonetheless may be obtained from their study.

Question 27

Quota Sampling

Answer 27

A method of forming a sample in which a prespecified number of individuals with certain background characteristics, such as a particular age, race, ethnicity, sex, or education, are selected for inclusion.

Often, participants are recruited as they arrive; once the quota for a given demographic group is filled, the researcher stops recruiting subjects from that group.

A researcher who uses this approach can obtain a final study sample that has the same proportional characteristics as the target population, enabling statistical testing to be performed on a subset of cases that is appropriately representative of the larger group of interest.