Lecture 27: Critical Thinking About Psychological Constructs

Question 1

What is the interpretation bias

Answer 1

The tendency to interpret the failure to confirm predicted outcomes in terms of method relevant beliefs, but confirmed predictions in terms of theory relevant beliefs

Question 2

What are theory relevant and method relevant beliefs

Answer 2

Theory relevant beliefs = theoretical mechanisms that produce observable behavior
Method relevant beliefs = procedures with which we produce and analyse data

Question 3

What does it mean when we say that method relevant beliefs are too peripheral and theory relevant beliefs are too central

Answer 3

Methods are too easy to disprove, and theories are too hard to disprove

Question 4

What are 3 common issues with model research practices that further exacerbate this

Answer 4

1. An over-reliance on conceptual replication
2. Problematic implementation of NHST = setting up an H0 as zero difference is a rather weak test of theory, there is a high chance that some differences will always be found —> straw target
3. Insufficient attention to verifying the integrity of measurement instruments and experimental procedures = psychological processes are context sensitive which makes validation of psychological measurement very difficult

Question 5

Why the interpretation problem a bigger issue in psych

Answer 5

Because measurements are not so robust and well established (as for example a thermometer would be, its very hard to prove that a thermometer is a wrongful measurement) as they are in other sciences

Question 6

What 3 things make it difficult to infer conclusions from surveys

Answer 6

1. Surveys are inherently subjective = people’s interpretations of questions can differ a lot
2. Survey measurement is context sensitive = the way and order in which questions are asked makes a difference for responses; social desirability/test-retest reliability is rarely checked
3. Conflation

Question 7

What is conflation

Answer 7

When two different things are measured that are actually influenced by the same constructs
Eg. a positive correlation was found between self-reported physical activity and literacy and numeracy, but maybe to be able to accurately self-report physical activity one has to already have a higher level of numeracy/literacy

Question 8

Which three kinds of validity need to be tested explicitly

Answer 8

1. Construct
2. Predictive
3. Test-retest reliability

Question 9

What does it mean when a metric is arbitrary

Answer 9

That the number on that metric don’t tell you much, only when put into context they become interpretable

Question 10

T/F: the midpoint of a scale always corresponds to the neutrality of a person on the construct that is being measured

Answer 10

False, this is not the case and something we need to look at explicitly and make sure we do not just assume

Question 11

What is meter reading

Answer 11

Using the score on the observed metric to infer location on the underlying dimension; just looking at the scores on the scale without investigating where the actual point of neutrality is

Question 12

What are 3 recommendations for strengthening method-relevant beliefs

Answer 12

1. Stronger emphasis on close replication
2. Verify integrity of methodological procedures
3. Use stronger forms of NHST

Question 13

What is a recommendation for weakening theory-relevant beliefs

Answer 13

Hypotheses under test should be such that their not being true is possible, so that their truth is contingent rather than necessary

Question 14

What is conservatism

Answer 14

Choosing the theoretical explanation consistent with the data that requires the least amount of restructuring of the existing knowledge system