Reliability, Validity & Inferential Statistics Flashcards

1
Q

What is reliability?

A

whether something is consistent in a study and whether it is replicable

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What is external reliability?

A

the consistency of measures over time, can be assessed using the test-retest method

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What is test-retest reliability?

A

involves presenting the participants with the same test/questionnaire on two separate occasions and seeing whether there is a positive correlation between the two

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What is inter-observer (or inter-rater) reliability?

A

the extent to which two or more observers are observing and recording behaviour in the same way - due to the issue of subjectivity in observational studies

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What is internal reliability?

A

refers to how consistently a method measures within itself - may use the split-half technique to see how similar they are

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What us the split-half technique?

A

half of the scores in a test are compared with the other half to see how similar they are - may be done by comparing odd scores on IQ test to the even scores, comparison seen by looking at how well the scores correlate

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

How do we improve reliability in an experiment?

A
  • controlling variables e.g. given standardised instructions as they can check the replicability of the findings
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

How do we improve reliability in observations?

A
  • observers must have clear and operationalised categories of behaviour and also trained in how to use the categories (may be shows video footage of similar behaviour they will observe)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

How do we improve reliability in content analyses?

A
  • use very tightly controlled and defined coding units and training beforehand
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

How do we improve reliability in questionnaires?

A
  • identify questions causing issues and rewrite/replace these questions to avoid the problem of subjective interpretation changing how they’re responded to
  • in general, closed-questions more reliable than open-questions
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

How do we improve reliability in interviews?

A
  • use same interviewer each time

- structures more reliable than unstructured

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What is validity?

A

whether something is true, and if it measures what it’s set out to measure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What is the internal validity?

A

in relation to experiments, whether the results were due to the manipulation of the IV rather than other factors such as extraneous variables or demand characteristics

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What is the external validity?

A

whether it is possible to generalise the results beyond the experimental setting

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

How do we asses the internal validity?

A
  • face validity

- concurrent validity

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What is the face validity?

A

simple way of assessing whether a test measures what it claims to measure which is concerned with face value - e.g. does an IQ test look like it tests intelligence

17
Q

How is face validity assessed?

A

by giving the test to experts in the field to look over, as they will know what is more likely to be relevant
- problem with it is that it is simply an opinion and there is no guarantee that it is correct

18
Q

What is the concurrent validity?

A

comparing a new test with another test of the same thing to see if they produce similar results, if they do then the new test has concurrent validity - if there is a strong positive correlation (i.e. exceeds 0.80+) between participants’ scores on the two tests

19
Q

What is ecological validity?

A

the extent to which findings of a research study are able to be generalised to real-life settings

20
Q

What is temporal validity?

A

refers to how likely it is that the time period when a study was conducted has influenced the findings and whether they can be generalised to other periods in time

21
Q

How do we improve the validity in experiments?

A
  • having a control group means the effect on the DV of the change in IV can be seen by comparing the results from the experimental condition to those from the control where the IV did not change
  • single blind procedures used to prevent the participants’ knowledge of which condition they are in from affecting their behaviour
  • double blind procedures
  • deception
  • standardised instructions
  • repeated measures design OR independent groups allocate to random conditions
  • with repeated measures, use counterbalancing to reduce order effects
22
Q

How do we improve the validity in observations?

A
  • behavioural categories need to be tightly operationalised so the observations are more objective
  • using covert observations
  • using naturalistic observations
23
Q

How do we improve the validity in self-reports?

A
  • validity of these threatened by participants not telling the truth due to social desirability bias or not understanding the questions properly
  • interviews mean participants can ask for the questions to be clarified
  • questionnaires decrease social desirability bias (researcher does not need to be present) especially if responses are kept anonymous
  • check for lies using a lie scale
24
Q

What is a lie scale?

A

same thing is asked about more than once, but in different ways and if answers are inconsistent this indicates participants have either misunderstood or lied

25
Q

How do we improve the validity in qualitative methods?

A
  • triangulation
26
Q

What is triangulation?

A

the use of a number of different sources as evidence, if all of these sources agree, interpretation is more valid

27
Q

What do we need to consider when deciding which inferential test to use?

A
  • design
  • difference
  • data (level of measurement)