Reliability Flashcards
(16 cards)
what is reliability?
The extent to which a text or measurement produces consistent results.
what does reliability refer to?
refers to consistency.
the same results should be obtained when …
a study is repeated using the same method, design and measurements.
what does internal reliability concern?
the extent to which something is consistent within itself
what does external reliability concern?
concerns the extent to which a test measures consistently over time
this is only correct for variable that stays the same overtime itself
what is the difference between internal and external reliability?
Internal reliability assesses the consistency of results across items within a test. External reliability refers to the extent to which a measure varies from one use to another.
what are the 2 ways of measuring reliability?
test-retest
inter-observer reliability
what is test-retest?
The test-retest method involves administering the same test or questionnaire to the same person but on different occasions.
To be considered reliable, the results from the test or questionnaire should be the same or very similar each time they are administered.
The 2 scores from the test or questionnaire will be correlated to make sure they are similar.
There must be sufficient time between test and retest to ensure that the ppt cannot recall their answer to the question, but not long enough that their opinion would change.
what is inter-observer reliability?
investigations should be conducted in teams of 2 to reduce subjectivity bias and unreliability in the data collection process. Observers need to record their data individually.
inter-observer reliability may involve … why does this need to be involved?
a small-scale run (pilot study) of the observation
A pilot study must be carried out to check that observers are applying behavioral categories in the same way or it may be reported that the data collected was reliable.
what happens to both sets of scores for test retests and the inter-observer method?
the sets of scores are correlated.
the correlation coefficient should exceed … for reliabilty
.80
how can the reliability of questionnaires be improved?
to test the reliability of questionnaires, the test-retest method should be used.
a questionnaire that produces low rest-retest reliability may require some of the items to be ‘deselected’ or rewritten.
complex and ambiguous open questions should be replaced with closed questions that are less ambiguous so less likely to be misinterpreted.
how can the reliability of interviews be improved?
the same interviewer should be used for each interview.
if this isn’t possible then the interviewers should be properly trained so that there answers aren’t too leading or ambiguous.
structured interviews should also be used where the interviewers behaviour is more controlled by the fixed questions.
how can the reliability of observations be improved?
by making sure that behavioural categories are operationalised, and that they are measurable and self-evident.
behavioural categories should not overlap and all possible behaviours should be covered on the checklist.
if categories are not operationalised well, different observers have to make their own judgements of what to record and so may end up with differing and inconsistent records.
how can the reliability of experiments be improved?
standardised procedures must be used to ensure that results can be compared.
