Reliability (test-retest, interobserver, improving reliability)

Question 1

2 Ways assessing Reliability

Answer 1

test-retest method, inter-observer reliability

Question 1

Reliability

Answer 1

Reliability refers to how much we can depend on a measurement or a set of findings. It provides a judgement of whether a measurement or set of findings are consistent. If you use a ruler to measure the height of a chair today and then do this again tomorrow, you expect the ruler to be consistent (reliable) and produce the same measurement. You would assume that any change in the height was because the chair had changed.

Any measuring device used in psychological research must be reliable. For example, a psychological test assessing personality or an interview about drinking habits must be reliable in order to be able to depend upon the results gathered. If it is measuring the same thing for each person each time, it should produce the same result on every occasion. Then if the result is different, we know that the ‘thing’ we are measuring (personality or drinking habits) must have changed.

Question 2

test-retest method

Answer 2

a study is repeated using the same participants. If similar results are obtained, then the findings are reliable. This is a common way to assess the reliability of psychological tests, questionnaires and interviews. It can also be used to assess reliability of the method used in experiments. In other words, whether the same procedures were used to measure the dependent variable for all participants. With this method, it is necessary to consider the amount of time that elapses between each test.

Question 3

inter-observer reliability

Answer 3

this is a way of assessing whether all researchers involved in a specific investigation are gathering consistent observations or ratings. If similar results are obtained by all researchers, then findings are reliable. This is a commonly used way to assess the reliability of data collected from studies using observational techniques.

Question 4

what is the way of assessing reliability in interviews and correlations

Answer 4

inter-rater reliability

Question 5

How reliability can be improved in experiments

Answer 5

*Use a standardised procedure. This will help ensure the same procedures are used to measure the dependent variable for all participants.

*If possible, take more than one measurement from each participant and record the average score.

Question 6

How reliability can be improved in questionnaires/ psychological tests

Answer 6

*Ensure questions are carefully worded – avoid leading or ambiguous questions that might persuade participants to provide different responses on different occasions.

*Replace open questions with closed questions as the responses to closed questions are more likely to be interpreted by researchers in the same way.

Question 7

How reliability can be improved in interviews

Answer 7

*Use the same interviewer for all participants to ensure all participants are studied in the same way.

*If it is not possible to use the same interviewer, ensure all interviewers are properly trained. This helps to avoid leading participants and also ensures that any rating system consistently.

*Ensure questions are carefully worded – if possible, use a structured interview. This reduces the risk that participants will interpret the same questions differently on different occasions.

Question 8

How reliability can be improved in Observational techniques

Answer 8

*Ensure behavioural categories have been operationalised and do not overlap or cause confusion.

*Behavioural categories should also cover all possible examples of the behaviour being studied.

*All observers should be trained to use a coding system. This ensures that all observers will use the coding system consistently.