lecture 24 (reasoning)

Question 1

intrepertation bias?

Answer 1

• bias towards interpretations that favour a researcher’s theory, both when getting significant results and when not
• identification bias in which part is false (theory or assumptions)
• increases error rates
• e.g.: As measure A is apparently a better measure of attention in this context, and B is less useful, there is some support for Enclothed cognition

Question 2

two different frameworks in research findings in psychology?

Answer 2

• method relevant beliefs (whatever you consider to be true about proper measurement)
• theory relevant beliefs (whatever you consider to be true about psychology/ human behavior)

Question 3

centrality and being peripheral?

Answer 3

• centrality of belief: belief on which many other beliefs depend -> peripheral is the opposite of being central
• a problem in psychology is that method-relevant beliefs are too peripheral and theory-relevant beliefs are too central (by this they mean that theories are often indistinguishable from very general assumptions about human behaviour)
• centrality of theory related beliefs + methods that are highly peripheral/easy to discard = higher likelihood of interpretation bias

Question 4

conservatism?

Answer 4

• the more central method-relevant beliefs, the more we are forced to be conservative in the interpretation of a study
• Preference for the interpretation that keeps established knowledge structures intact as much as possible
• this constrains the field of alternative explanations and so makes empirical tests more diagnostic

Question 5

fundamental problems in model research practice (MRP)?

Answer 5

1. overemphasis on conceptual replication
2. NHST implementation problems
3. lacking attention given to verifying measurement instruments’ and experiments’ integrity

Question 6

overemphasis on conceptual replications?

Answer 6

• weakens method-relevant beliefs and contributes to the constant theoretical advancement in MRP, however with faulty results
• leaves it ambiguous whether the results are due to theory-, or methodology-related circumstances
• Failure to produce significant results
  constitute failed pilot studies that end up in the file drawer, which increases the type I error in general and furthers publication bias

Question 7

Problematic implementation of NHST?

Answer 7

• all tyhe problems also listed in lecture 23 (false dillema/straw man fallacy…)
• encourages one-sided views on theories instead of more nuanced ones

Question 8

Not verifying the integrity of measurement instruments?

Answer 8

• not including reliability measures, not doing manipulation checks, or not replicating known effects
• weakens method-relevant beliefs
• Psychological processes are context sensitive which makes validation of psychological measurement very difficult

Question 9

Surveys?

Answer 9

• more direct observable method
• inherently subjective: interpretation and phrasing might differ between respondents
• survey measures are context dependent: social desirability bias, order of the questions, vagueness and ambiguity of language, etc
• young children’s self-reports, especially with parental help, may be inaccurate due to limited understanding and parental influence on responses
• reliability and validity of surveys not necessarily of how well questions measure construct, but how well questions within a survey correlate to eachother

Question 10

conflation?

Answer 10

• assuming you are measuring two different things that correlate but you are actually measuring the same construct or overlapping constructs
• e.g. children who are better at counting are better at reporting their physical activity levls because that involves counting

Question 11

Implicit association test (IAT)?

Answer 11

• design to surprass subjectiveness and social desirability bias of surveys
• IAT scores reflect differences in average reaction times

Question 12

validity and reliability of IAT?

Answer 12

• predictive validity: IAT scores are weak/moderate predictor of discriminatory behavior
• construct validity: lack of consistency with explicit measures of prejudice -> If an IAT measures implicit attitudes, why do most men not exhibit “implicit sexism”?
• test-retest reliability: IAT scores turn out to be a poor predictor of future scores by the same individuals on the same test

Question 13

issue of arbitrary metrics?

Answer 13

• assumes that response times in milliseconds directly reflect implicit biases without linking scores to meaningful behaviors
• meter reading: assuming that raw scores on a psychological metric represent a direct position on a psychological dimension (does 0 actually equal neutrality?)
• if you want to give a diagnostic interpretation to people’s score you need to look beyond just the positive correlation -> you should look at behavior neutrality (which in IAT equals 0.5)
• norming: Transforming raw scores into standardized scores or percentiles, which still may not reflect real-world meaning

Question 14

big take away implicit measures?

Answer 14

• Implicit measures (like the IAT) do not necessarily reflect our ‘true’ preferences
• Measuring blood oxygenation isn’t
  strictly speaking a more ‘direct’ measurement of psychological states than reaction time measures or even survey questions

Question 15

strategies for improving MRP?

Answer 15

• Direct replications should be emphasized more than conceptual replications (this ensures stronger underlying methodology)
• integrity of manipulations and measurements should be verified
• Null hypotheses should not be framed as no difference but should indicate a direction so that stronger claims can be made based on the results

Question 16

what can we conclude if we compare fMRI measurements in research into certain psychological states with measurements based on self-report questions about the same psychological states?

Answer 16

Both types of measurement of the same psychological state can only be valid if they are correlated