Reliability

Question 1

What is reliability?

Answer 1

• Refers to how consistent a measuring device is
• If measurement is made twice and produces same result= reliable

Question 2

What are the main ways of assessing reliability?

Answer 2

• Test-retest reliability
• Inter-observer reliability

Question 3

What is test-retest reliability?

Answer 3

• Administering same test to same person on different occassions
• If test is reliable, results should be the same/very similar both times
• Commonly used with questionnaires and psych tests
• Must be sufficient time between test and restest to ensure answers cannot be recalled but attitudes/ opinions/abilities haven’t changed

Question 4

What is inter-observer reliability?

Answer 4

• The extent to which there is agreement between two or more observers
• May involve pilot stidies to check observers apply behavioural categories in the same way
• Watch same event, record data independently

Question 5

How is reliability measured?

Answer 5

• Using correlational analysis
• Two sets of scores are correlated
• The correlation coefficient should exceed +.8

Question 6

How to improve reliability of questionnaires?

Answer 6

• Require some items to be deselected/rewritten (low test-retest)
• Reduce ambiguity or complexity of questions
• Replace some open questions with closed, fixed-choice alternatives

Question 7

How to improve reliability of interviews?

Answer 7

• Interviewers must be properly trained (not ask leading questions)
• Use of structured interviews instead, so behaviour is more controlled through use of fixed questions

Question 8

How to improve reliability of observations?

Answer 8

• Ensure behavioural categories have been properly operationalised, so are measurable and self-evident
• Categories should not overlap
• Further training for observers
• Discussion of decisions

Question 9

How to improve reliability of experiments?

Answer 9

• Ensure procedures are consistent each time
• Standardisation