Reliability Flashcards
What is reliability
measure of consistency allowing ability to replicate
Discuss test-retest reliability
test-retest reliability;
way to asses the reliability of a test/questionnaire
administer the same test-questionnaire to the same person/people on different occasions and if the test/questionnaire is reliable then the results obtained on each occasion should be the same or at least very similar
What is important to consider when assessing reliability with test-retest
there must be sufficient time between the test and retest to ensure that Pp cannot directly recall their answers from the previous survey but also not long enough that their attitudes/opinions/abilities may have changed
How can you measure the similarity between the answers in test-retest
use a correlational test to compare the two scores
if the correlation is significant and positive then the reliability of the measuring instrument is assumed to be good
Discuss inter-observer reliability
used to assess reliability in observational research as one researchers interpretation of events may differ to another’s which would introduce subjectivity, bias and unreliability into the data collected therefore this ensures that categories are appropriately operationalised
may involve a pilot study to check that observers are applying behavioural categories in the same way or a comparison of collected data at the end of a study
the extent to which observers agree on the collected data of behaviours from the same observation using the same behavioural categories; measured by correlating the observations of observers
(+8.0 = high inter-observer reliability)
How can you improve reliability in observations
ensure that sufficient training of observers takes place so that they can accurately assign particular observed behaviour to the correct behavioural categories
make sure that behavioural categories have been operationalised to make sure they are measurable and self-evident
How can you improve reliability in interviews
use structured interviews rather than unstructured; this will mean interviewers will use a script and ensures each Pp will have a similar experience and provide answers allowing comparison
ensure sufficient training takes place so that interviewers do not ask leasing or ambiguous questions
How can you improve reliability of questionnaires
use test-retest method and if it produces low test-retest reliability then perhaps some items need to be deselected or rewritten as they might be too ambiguous leading to multiple interpretations
use closed questions rather than open ones to reduce the range of possible answers
How can you improve the reliability of experiments
use standardised procedures for each Pp for example what is said to each Pp is kept the same as well as the environmental conditions
