Research Methods- Reliability

Question 1

Reliability:

Answer 1

Saying results are reliable is another way of saying the results are consistent.

If the researchers replicate their study exactly, they will get similar results.

Question 2

Psychologists use a range of measurement tools to collect data from participants:

Answer 2

• Questionnaires & Interviews: Use set lists of closed and/or open questions to measure opinions, personality traits and recalled past behaviours.

•Experimental conditions: Use a range of measures when testing participants between conditions. Experiments include biological devices such as EEG, cognitive tests such as the Stroop test, or entire physical environments like skinner boxes.

• Observations: Use clearly defined lists of operationalised behavioural categories to classify observed behaviours.

Question 3

External reliability:

Answer 3

The extent to which a measure is consistent when repeated. (e.g. the results of a study are consistent with an exact replication at a different time and/or with different participants)

Question 4

Internal Reliability:

Answer 4

The extent to which different parts of a measure are consistent with itself. (e.g. if a 100-question IQ test is divided into two 50-question tests, the results for each set of questions with the same participant would be similar)

Question 5

Assessing Internal Reliability

Answer 5

Split half method: Assess measures that test one variable with multiple questions, for example, questionnaires or tests of IQ:

• Split the test into two parts.
• Participants complete both parts.
• Test the strength of the correlation between the two parts of the measure.
• A strong correlation indicates internal reliability

Question 6

Assessing External Reliability

Answer 6

Test-retest: Repeat the study using the same procedures / measuring devices at different times and test the correlation between the two versions.

Inter-observer/ inter-rater reliability: Two (or more observers record behaviours during the same observation using the same behavioural categories; then, they test the correlation between each tally of behaviour to identify if the behavioural categories are appropriately operationalised.

Comparing with a test of correlation: The level of correlation is assessed using a test of correlation such as Pearson’s R or Spearman’s Rho. A correlation of 0.8 or higher is usually accepted as a strong correlation.

Question 7

Improving reliability

Answer 7

Changes psychologists can make to the design of their studies

Question 8

Improving reliability
Observation

Answer 8

Improving the training given to observers can improve their accuracy in assigning a particular observed behaviour to the correct behavioural category. Pilot studies can identify poorly defined behavioural categories; operationalisation is the process of clearly defining a behavioural category.

Question 9

Improving reliability
Interviews:

Answer 9

Use structured interviews rather than unstructured interviews. The interview will then include a script the interviewer can follow, ensuring each participant has a similar experience and provides answers. The researcher can then compare responses from each participant.

Question 10

Improving reliability
Questionnaires:

Answer 10

Questionnaires: Use closed questions to reduce the range of possible responses, and if there is an established questionnaire that tests for what you need to measure, use that rather than creating a new test.

Question 11

Improving reliability
Experiments:

Answer 11

Experiments: Use standardised procedures for each participant, so they all have the same experience. For example, keep what is said to the participants the same (by using a script) and environmental conditions the same. Use established tests as measures rather than creating a new test.