Selecting the best Measure

Question 1

What is one of the difficult things about measurement in psychology?

Answer 1

We need operational definitions

Question 2

What are some observable measures we can choose to look at?

Answer 2

• Verbal response
• Nonverbal response
• Physiological response
• Overt actions

Question 3

What are the 2 elements that are contained in every measurement?

Answer 3

“True” score-hypothetical concept
Error
“True” score+ error=observed score

Question 4

What are 3 sources of measurement error?

Answer 4

Experimenter, participant, observer/scorer

Question 5

What are 2 sources of experimenter error?

Answer 5

Random Error: Time of day, temperature, noise.

Bias: Experimenter characteristics, experimenter expectancies.

Question 6

What do we mean by experimenter characteristics?

Answer 6

When a particular aspect of the experimenter affects how participants respond-can be physical characteristics (age, gender, ethnicity) OR personality (friendliness, hostility, anxiety).

Question 7

How do we control for experimenter characteristics?

Answer 7

Use standardized methods- train experimenters to follow set standards when administering procedures, standardize aspects of experimenter as much as possible (appearance, attitude, etc)
Replication!

Question 8

What do we mean by experimenter expectancies?

Answer 8

When the expectations of the experimenter affect how the participant behaves. Not limited to humans!

Question 9

What are 2 examples of experimenter expectancies?

Answer 9

• The Rosenthal effect (educational “Bloomers”

- Maze bright versus maze dull rats.

Question 10

How do we control for experimenter expectancies?

Answer 10

• Standardization (instructions scripted, recorded in advance, or presented via computer).
• Objectivity (make coding schemas as objective as possible, automated recording equipment)
• Single-blind research

Question 11

What are some participant errors?

Answer 11

Random: Carelessness, distraction
Bias: Demand Characteristics, good participant effect, response bias

Question 12

What are demand characteristics?

Answer 12

Features of an experiment that seem to inadvertently cause participants to act in a certain way.

Question 13

What is the good participant effect?

Answer 13

Tendency for participants to behave as they perceive the researcher wants them to behave.

Question 14

What is it called when demand characteristics and the good participant effect work together?

Answer 14

Pact of Ignorance (Orne 1968).

Question 15

How do we control for demand characteristics?

Answer 15

Conduct double-blind research

Deception

Question 16

What is response bias?

Answer 16

When the context affects the way a participant responds. (yes sayers vs nay sayers). Social desirability is also an issue.

Question 17

How do we control for response bias?

Answer 17

Include both “agree” and “disagree” items.
Randomize question presentation
Careful review of questions/setting
Pilot testing

Question 18

What are some types of observer error?

Answer 18

Random: carelessness, distraction

Observer/scorer bias: Confirmatory bias (see what we want to see).

Question 19

How do we control for observer error?

Answer 19

Eliminate human observer-use mechanical measure instead
Limit observer subjectivity-focus on observable behaviour, use standardized coding schema
Make observer “blind”-unaware of experimental condition

Question 20

What is construct validity?

Answer 20

The extent to which your manipulation or measure actually represents the claimed construct (ex: does your measure of extraversion actually measure obnoxiousness?)

Question 21

What are some criteria needed for construct validity?

Answer 21

1) Reliability
2) Content validity
3) Convergent validity
4) Discriminant or divergent validity

Question 22

What is reliability?

Answer 22

The repeatability or consistency of the research

Question 23

What is test-retest reliability?

Answer 23

Comparable scores on retest. Relationship between scores at time 1 and time 2. The difference between the observed value and 1 equals the proportion of random error in scores.

Question 24

What is inter-rater reliability?

Answer 24

Comparable scores between observers. Calculated the same way as test-retest, and then percentage agreement.

Question 25

What is internal consistency?

Answer 25

The extent to which responses to items that propose to measure the same construct are similar. Variability across items may be due to random error or more than one construct being assessed.

Question 26

How do we test internal consistency?

Answer 26

Average inter-item correlation Split-half correlation Cronbach's Alpha

Question 27

How do we improve internal consistency?

Answer 27

Add items/questions-random error balances out. Create better items/questions (reduces potential variability in interpretation)

Question 28

What is content validity?

Answer 28

The extent to which a measure covers all aspects of a construct. Ex) measuring love measures commitment, sexual attraction, liking.

Question 29

How do we make sure something measures all aspects of a construct?

Answer 29

Theory, definitions, experts. Make sure to measure all component dimensions, have a large enough set of measures for each dimension.

Question 30

What is convergent validity?

Answer 30

The extent to which a measure correlates with other indicators of the same construct. ex) People scoring high on your measure should also score high on other measures of the same construct.

Question 31

How do we asses convergent validity?

Answer 31

Similar measures, known comparison groups, other indicators of construct.

Question 32

What is discriminant validity?

Answer 32

The extent to which your measure is distinguishable from related constructs. People scoring high on your measure should not be scoring as high on measures of similar constructs.

Question 33

What is discriminant validity?

Answer 33

The extent to which your measure is distinguishable from other constructs (unrelated constructs). People scoring high on your measure should not also score high on measures of the "wrong" construct.

Question 34

What else do you need in a research study.

Answer 34

Best measure. Best fit for the research context. Additional issues-is the scale appropriate to the context? Is the measure sensitive enough?

Question 35

What is sensitivity?

Answer 35

Ability of measure to detect effects. Is the measure strong enough for what you want to study? Does your measure minimize the influence of error?

Question 36

How do we achieve sensitivity in measurement?

Answer 36

Use measure with score variance (avoid restriction of range) Avoid all or nothing measures (ask how much instead( Add scale points to rating scale Pilot test measure.