Scientific Methods Flashcards

1
Q

Pre-Scientific Methods?

A

Astrology
- Personality assessment based on birth date

Physiognomy
- Personality assessment baed on shape of body, particularly the face

Phrenology
- Personality assessment based on morphology (shape) of skull

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

4 major types of descriptive methods?

A

LOTS of data!

Life history data
Observer-reports
Test data
Self-reports (surveys)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Self-Report?

A

Asking people questions about their beliefs and behaviours

  • Provided by the participant
  • Responses to questionnaires
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Ten-Item Personality Inventory-(TIPI)?

A

I see myself as: (rate on a likert scale)
1. _____ Extraverted, enthusiastic
2. _____ Critical, quarrelsome.
3. _____ Dependable, self-disciplined.
4. _____ Anxious, easily upset.
5. _____ Open to new experiences, complex.
6. _____ Reserved, quiet.
7. _____ Sympathetic, warm.
8. _____ Disorganized, careless.
9. _____ Calm, emotionally stable.
10. _____ Conventional, uncreative.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Self-Report Data, advantages and disadvantages?

A

Advantages
– Allows study of difficult-to-observe behaviors,
thoughts and feelings
* Who knows better?
– Easy to distribute to large groups

Disadvantages
– Respondents may not be representative
(convenience sampling is tempting)
– Responses may be biased or untruthful

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Observer report?

A

Observing behaviour of others

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Observer Reports:
Who are the Observers?

A
  • Parents, friends, teachers
    – Usually collected by questionnaire or rating
    form
  • Trained observers
    – Systematic observations of behavior
  • Untrained, participant-observers
    – Class ratings of Trudeau
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Observer report, advantages and disadvantages?

A

Advantages
– Capture spontaneous behaviors
– Avoid bias of self-reports

Disadvantages
– Researcher interference
* How naturalistic (vs. artificial) is the observation?
– Rarity of some behaviors
* Research on criminality
– Observer bias & selective attention
– Time consuming

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Test Data?

A

Assessing an individual’s
abilities, cognitions, motivations,
or behaviors, by observing their
performance in a test situation
Tests may be written,
physical (e.g.,
cardiogram),
experimental, or
physiological

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Examples of Kinds of Test Data?

A

Questionnaire tests
– E.g., IQ

Experimental tests
– Megargee (1969) study of dominance
* Does trait dominance (high vs. low) or gender
predict leadership?
* Paired high and low dominant men and women in
“box repair” task
* 4 kinds of groups:
(1) high dom ♀, high dom ♂
(2) high dom ♀, low dom ♂
(3) low dom ♀, low dom ♂
(4) low dom ♀, high dom ♂

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Test data, advantages and disadvantages?

A

*Advantages
* Allows measurement of characteristics
not easily observable, or known to the
participant

Disadvantages
* Must infer that the test measures what
you think it measures
– Validity issue

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Life history (Case studies)

A

Intensive examination
of a single person or
group

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Case Study Methods?

A
  • Obtained from life history (interviews,
    autobiography)
  • Other life records (Life Outcome Data)
    – School grades
    – Criminal records
    – Work record
    – Facebook page, tweets, instagram, etc.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Case study, advantages and disadvantages?

A
  • Advantages
    – Rich source of hypotheses
    – Allows for studies of rare behaviors
  • Disadvantages
    – Observer bias
    – Difficult to generalize (N = 1)
    – Difficult to reconstruct causes from complexity
    of past events
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Reliability?

A

Extent to which scores on the measure
are stable and replicable, vs. amount of
error or randomness in the measure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Validity?

A
  • Degree to which measure assesses what it is
    supposed to assess
17
Q

Bulls eye analogy?

A
  • Reliability = are you hitting the same spot
    each time?
  • Validity = are you actually hitting the bulls
    eye?
18
Q

Measuring Validity (4)?

A
  • Face validity
    – Does it measure what you think it measures?
  • E.g., shyness questionnaire
  • Predictive validity
    – Does it predict an external criterion?
  • Does shuffling predict self-reported shyness?
  • Convergent validity
    – Relation to other measures of same variable
  • Self-report and observer report should be related
  • Construct validity
    – All of the above
19
Q

Inter-rater Reliability and Validity
An Example: Measuring Height (without a ruler)

A
  • Reliability of ratings of height
    – Average correlation between two judges = .76
    – Reliability of 5 judges is about .90
  • Validity can only be high if reliability is high
    – If measures are more reliable, they provide a more valid
    assessment!
    – By combining the judgments of multiple people (or using
    multiple items on a personality test) we can get fairly reliable and valid measures of personality (reliabilitiesnabout .80-.90)
20
Q

The Problem:
Incentives Structure?

A
  • Published research is important for
    getting a job, getting tenure, getting
    grants, and being viewed favorably in
    our field
  • Result: scientists try to publish as
    much as they can
  • Balancing act: need to stay truthful to
    psychological science, but also
    publish
  • This results is researchers taking
    shortcuts and sometimes worse…
21
Q

Questionable Research Practices
(QRPs)?

A

Decisions in design, analysis, and
reporting that increase the likelihood of
achieving a positive result
– And a positive response from editors and
reviewers

22
Q

What should researchers do to avoid QRPs?

A
  • Increase disclosure in methods,
    results, and hypothesis presentation
  • Pre-register hypotheses and studies
    – Data collection rules, analytic strategies
  • Share data
  • Be a responsible scientist regardless of
    outcome
23
Q

Center for Open Science?

A

Founded to increase to openness,
integrity, and reproducibility of scientific research
– Brian Nosek and Jeff Spies
* Open source software platform for
pre-registering hypotheses,
archiving study materials,
depositing data and syntax

24
Q

Good Research?

A
  • Good research is open research
    – Materials and data are shared publicly
  • Good research features experimental
    methods that are strong and isolate a
    question of interest
  • Good research is adequately
    “powered” research
25
Q

Power?

A
  • Most psychological effects are small, so
    you need a lot of participants
    – Some say “around 200”; others say it depends
    on what you’re studying and your design
    – If you’re studying an effect that’s likely to be
    small, you need a big sample
  • E.g., Are UBC or SFU students more liberal?
    – If you’re studying an effect that’s likely to be
    big, a smaller sample is ok
  • E.g., Are UBC students or Texan oil tycoons more
    liberal?
26
Q

What does it mean that Power is generally set at 80%?

A

– This means there’s an 80% chance of
finding an effect that exists
– However, studies are often run with much
lower power
* Researchers underestimate how much data
are needed
* Effects are smaller than they think
* It’s hard and expensive to collect large
samples