Validity And Reliabilty Flashcards
Reliability
This is the extent to which a test produces a consistent findings every time it is done
We need to be able to measure or observe something time after time and produce the same or similar results-reliability shows consistency over time.
How to asses reliability
- Inter-observer reliability: use a team of at least 2 observers to gain objective, reliable results.
• Carry out pilot study for training purposes to ensure observers can apply the behavioural categories in the same way.
• Correlate results at the end of the observation to assess reliability.
• Usually looking for a correlation co-efficient of around +0.80-1.00
How else do we test it
- Test-retest method – Compares a participant’s performance on the same people on different occasions (external reliability).
•You can also do a correlation between the two sets of scores.
•- High correlation co-efficient indicates that the test is reliable.
•- A low-correlation coefficient indicates poor reliability.
Improving reliability
•Where observer scores do not significantly correlate then reliability can be improved by:
•Training observers in the observation techniques being used and making sure everyone agrees with them.
•Ensuring behaviour categories are correctly and objectively operationalised. This means that the behaviour being observed can only be that behaviour. For example, “aggressive behaviour” is subjective and not operationalised, but “pushing” is objective and operationalised.
How to measure it
• The test-retest method involves
• administering an entire test to a participant, waiting for them to ‘forget’ the questions (which could take several weeks), and then re-administering the test.
• If the results from both presentations of the test significantly positively correlate then it is a reliable test.
• If results don’t correlate then rewriting some questions may be needed.
• The disadvantages of the test-retest method are that it takes a long time for results to be obtained, and if too long an interval has been used then the participant may have changed in themselves which may mean a test is declared unreliable when it is in fact reliable.
• The advantage is that every question is checked for reliability.
Validity
Refers to credibility or legitimacy of a measurement of behaviour.
Does it look like it measures what it says? Face
Comparing a new test with an existing test (of the same nature) to see if they produce similar results.? Concurrent
Does it stand the test of time? Tempora
Are results representative outside of the lab setting? Ecological
Are techniques used to collect data in tests, questionnaires, interviews and observations measuring what is claimed?
.
• Validity is the extent to which a test measures what it claims to measure.
• For example does an IQ test really measure your intelligence or does a personality test really measure your personality?
• When we measure behaviour in a laboratory are we measuring the same behaviour as in real life.
• When we measure behaviour on a questionnaire are people telling us the truth or are we just measuring what they think we want them to tell us?
Concurrent validity
Compares new measurement with an existing, validated one.
Check new measure test scores are correlated with those from an established valid test.
Concurrent validity is demonstrated when a test correlates well with a measure that has previously been validated e.g. IQ tests.
Expect correlation of +0.80 or higher
Temporal
Temporal Validity
This refers to how relevant the time period is in affecting findings
E.g. would findings from a study on attitudes to romantic relationships in 1950 apply to attitudes today ?
If not, then it has low temporal validity.
Factors affecting validity
• Bias and operationalising (measuring) your variable.
• So whether it is an experiment, an observation or a self report if you want to improve
your validity you need to make sure you are measuring your variable accurately and that you are reducing any bias like experimenter bias, observer bias, demand characteristics or social desirability bias.
• To what extent are the results due to the manipulation of the IV? Extraneous variables c
need to be controlled otherwise they can become confounding variables which may
affect reliability and therefore validity. • Use single or double blind studies
Improving validity
• no Bias and operationalising (measuring) your variable.
• So whether it is an experiment, an observation or a self report if you want to improve
your validity you need to make sure you are measuring your variable accurately and that you are reducing any bias like experimenter bias, observer bias, demand characteristics or social desirability bias.
• To what extent are the results due to the manipulation of the IV? Extraneous variables c
need to be controlled otherwise they can become confounding variables which may
affect reliability and therefore validity. • Use single or double blind studies
