PSYC*2360 Chapter 4: Tools for Building Better Designs

Question 1

Which research design involves manipulating all independent variables and randomly assigning participants to groups?

Answer 1

True experiments

Question 2

What is a quasi-experimental design?

Answer 2

A research design in which the researcher can’t manipulate the independent variable or use random assignment

Question 3

Are non-equivalent, posttest only designs and non-equivalent, pretest-posttest designs experimental or quasi-experimental?

Answer 3

Quasi experimental

Question 4

What is the difference between non-equivalent group, posttest only and non-equivalent group, pretest-posttest designs?

Answer 4

• Posttest only: Outcome measures are administered to two or more groups without random assignment
• Pretest-posttest: Baseline measures of groups are taken before an intervention, then an outcome measure is administered

Question 5

What are quasi-independent variables?

Answer 5

Variables treated as if they are independent variables in an experimental design, even though they can’t be manipulated

Question 6

What is self-report?

Answer 6

Any measurement technique that directly asks participants how they think or feel

Question 7

What are the two most common examples of self-report measures?

Answer 7

• Interviews
• Surveys/questionnaires

Question 8

What are the two main advantages of self-report measures?

Answer 8

• Relatively cheap and easy to administer
• Receive first-hand information from the source

Question 9

Social desirability, demand characteristics, and retrospective bias are the main drawbacks of which type of measure?

Answer 9

Self-report

Question 10

What is social desirability?

Answer 10

The tendency for respondents to give answers that make them look good

Question 11

What are demand characteristics?

Answer 11

Cues that potentially make the participants aware of what the researcher expects

Question 12

What is retrospective bias?

Answer 12

The tendency to view or interpret past events inaccurately

Question 13

What must researchers do to properly measure behaviour?

Answer 13

They must find a way to quantify it

Question 14

What are the three main types of behavioural measures?

Answer 14

• Behavioural trace
• Behavioural observation
• Behavioural choice

Question 15

What is a behavioural trace?

Answer 15

A behavioural measure that relies on evidence left behind by a participant who is longer present to determine if a behaviour occurred

Question 16

What are the two main advantages of behavioural trace measures?

Answer 16

• Doesn’t require the participants’ cooperation
• Avoids issues like social desirability and demand characteristics

Question 17

What is the main disadvantage of behavioural trace measures?

Answer 17

The interpretation of a behavioural trace’s meaning depends on the researcher to draw inferences

Question 18

What do behavioural observations rely on?

Answer 18

Directly seeing or observing behaviour as it occurs

Question 19

T or F: A drawback of behavioural observation is that it cannot go beyond what can be visually observed.

Answer 19

False

Question 20

Which behavioural measure involves participants making a purposeful selection from several options?

Answer 20

Behavioural choices

Question 21

What are the two main disadvantages of behavioural measures?

Answer 21

• They are often time-consuming, expensive, and cumbersome
• Introduce the possibility for participant reactivity and demand characteristics

Question 22

What is participant reactivity?

Answer 22

When participants act differently or unnaturally because they know someone is watching them

Question 23

T or F: Behavioural measures tend to provide findings that are more relatable to the general public.

Answer 23

True

Question 24

What are unobtrusive measures?

Answer 24

Strategies that allow for observation and assessment without a participant’s awareness

Question 25

What is the difference between a raw scare and a true score?

Answer 25

- Raw score: The actual score, comprised of a true score and error - True score: What the raw score would be if the test were a perfect measure and was uninfluenced by extraneous factors

Question 26

What are the extraneous influences that cause a raw score to deviate from the true score referred to as?

Answer 26

Errors

Question 27

A variation from the measure's true score due to unsystematic or chance factors that occur in a haphazard, disorganized, or arbitrary way is known as what?

Answer 27

Random error

Question 28

T or F: It is possible to remove random error from a study.

Answer 28

False

Question 29

T or F: Bias is also known as a systematic error.

Answer 29

True

Question 30

What is bias?

Answer 30

An error that systematically pushes scores in one direction

Question 31

T or F: Statistics can be used to address bias.

Answer 31

False

Question 32

Which form of error is more problematic, random error or bias?

Answer 32

Bias

Question 33

What is standardization?

Answer 33

Keeping experimental conditions the same for everyone and as free from variation as possible

Question 34

T or F: Standardization helps minimize bias, but can't remove random error.

Answer 34

True

Question 35

What is observer/scorer bias?

Answer 35

The misinterpretation of an observation based on existing beliefs, previous experiences, or expectations

Question 36

What does the sensitivity of a measurement refer to?

Answer 36

The range of data a researcher can gather from a particular instrument

Question 37

To strike the right balance between being overly sensitive and not sensitive enough, it is recommended to use how many options in self-report measures?

Answer 37

5-9

Question 38

What is the ceiling effect?

Answer 38

When all responses of a measure fall at the top end of the scale

Question 39

In terms of the boundaries of a measurement tool, what might cause a ceiling effect?

Answer 39

The upper boundary being set too low

Question 40

If the independent variable is unable to affect the dependent variable, which is more likely, a ceiling effect or a floor effect?

Answer 40

A ceiling effect

Question 41

What is the floor effect?

Answer 41

When all responses of a measure fall at the bottom end of the scale

Question 42

In terms of the boundaries of a measurement tool, what might cause a floor effect?

Answer 42

The lower boundary being set too high

Question 43

What is the difference between reliability and validity?

Answer 43

- Reliability: The extent to which a measure will produce the same results each time it is used - Validity: The extent to which a research method measures what it is intended to measure

Question 44

Does random error threaten reliability or validity?

Answer 44

Reliability

Question 45

Does bias threaten reliability or validity?

Answer 45

Validity

Question 46

T or F: Without validity, there is no reliability.

Answer 46

False. Without reliability, there is no validity.

Question 47

T or F: Even with reliability, a measure can either be valid or invalid.

Answer 47

True

Question 48

What is the entire group of interest in a study referred to as?

Answer 48

The population

Question 49

What is the subset of a population from which the researcher collects data referred to as?

Answer 49

A sample

Question 50

What is the explicit strategy used for recruiting participants from the population referred to as?

Answer 50

The sampling plan

Question 51

What is a representative sample?

Answer 51

A sample with specific features that characterize the population of interest

Question 52

What is probability sampling?

Answer 52

A sampling approach in which everyone from a given population has an equal chance of being selected to participate in a study

Question 53

What are three types of probability sampling?

Answer 53

- Simple random sampling - Stratified random sampling - Cluster random sampling

Question 54

What is the procedure for simple random sampling?

Answer 54

A subset of individuals is randomly selected from the entire population

Question 55

T or F: Access to the entire population is not necessary for simple random sampling.

Answer 55

False

Question 56

What is the procedure for stratified random sampling?

Answer 56

The population is divided into strata (subpopulations), then simple random sampling is used to select participants from each stratum in proportion to the population at large

Question 57

What is the procedure for cluster random sampling?

Answer 57

The population is divided into groups/clusters, then simple random sampling is used to select which clusters participate

Question 58

T or F: In cluster random sampling, clusters are normally naturally occurring.

Answer 58

True

Question 59

What is non-probability sampling?

Answer 59

A sampling approach in which everyone from a given population does not have an equal chance of being selected to participate in a study

Question 60

What are four types of non-probability sampling?

Answer 60

- Convenience sampling - Quota sampling - Purposive sampling - Snowball sampling

Question 61

What is the procedure for convenience sampling?

Answer 61

Involves nonrandom selections of participants who are readily available to the researcher

Question 62

What is the procedure for quota sampling?

Answer 62

Researchers choose ahead of time that they want their sample to hold certain characteristics, then freely choose any participant that helps meet the quota

Question 63

What is the procedure for purposive sampling?

Answer 63

The researcher chooses a sample based on whom they think would be appropriate or qualified for the study

Question 64

Which type of non-probability sampling is most often used when a limited number of people have expertise in the area under investigation?

Answer 64

Purposive sampling

Question 65

What is the procedure for snowball sampling?

Answer 65

Existing study participants recruit future participants from among their acquaintances

Question 66

How does snowball sampling introduce bias into a sample?

Answer 66

The more social connections a person has, the higher their chance of selection

Question 67

What does non-response bias refer to?

Answer 67

The chance that those who refused to participate may systematically differ in a meaningful way from those who did

Question 68

Comparing the results of individuals who refused to participate at first, then agreed, with individuals who initially accepted, may be helpful when making inferences about which type of bias?

Answer 68

Non-response bias

Question 69

What is the volunteer subject problem?

Answer 69

The concern about the generalizability of conclusions from volunteer participants because they may differ from non-volunteers in meaningful ways

Question 70

What is the college sophomore problem?

Answer 70

The concern of generalizability, given that researchers often use convenience samples consisting of college/university students