Assessing Improving Reliability Validity

Question 1

Validity Overview

Answer 1

• Research should have validity - measuring what you think you are measuring.

Question 2

Internal Validity:

Answer 2

Internal = inside the study. Is the research measuring what it intends to measure? Is it measuring the effect of just the IV on the DV?
Affected by extraneous variables e.g. demand characteristics, researcher bias, investigator effects, social desirability, order effects.

Question 3

External Validity:

Answer 3

External = outside the study. Whether findings can be generalised outside the study.

Question 4

Ecological Validity

Answer 4

A form of external validity. Extent findings can be generalised beyond the setting of the study to other real life settings.

Question 5

Population Validity:

Answer 5

A form of external validity. Extent findings can be generalised beyond the sample studied to the target population.

Question 6

Temporal Validity:

Answer 6

A form of external validity. Extent findings remain true over time and can be generalised to other time periods.

Question 7

Face Validity:
Assessing validity

Answer 7

An independent psychologist in the same field looks at the experimental conditions (questions or behaviour categories) to see if they look like they measure what they intend to measure (CONTEXT). If yes, research has face validity.

Question 8

Concurrent Validity:

Assessing validity

Answer 8

Compare results of new test (CONTEXT) with results from a similar established test using stats test. Correlation should exceed +0.8. If similar, test is valid.

Question 9

Improving Validity

Answer 9

• Experimental Research:
Use control group to check IV affects DV (cause & effect).
Use standardised procedures/instructions to reduce investigator effects.
Use single blind to reduce demand characteristics & double blind to reduce demand characteristics and investigator effects.

Question 10

Validity in Observations
Face Validity:

Answer 10

Independent psychologist checks if behaviour category (CONTEXT) looks like it measures what it claims to measure at first sight.

Question 11

Concurrent Validity:
Validity of observation

Answer 11

Compare new observation with similar established observation, correlation should exceed +0.8.

Question 12

• Improving Observations:Validity

Answer 12

Behaviour categories operationalised.
Observers trained to use categories.
Use covert observations for natural behaviour.

Question 13

Validity in Self-Reports

Answer 13

Face Validity:
Independent psychologist checks if questions in questionnaire/interview (CONTEXT) look like they measure what they intend.

Question 14

Concurrent Validity:self report

Answer 14

Compare new questionnaire/interview with established one, correlation should exceed +0.8.

Question 15

Improving Self-Reports: validity

Answer 15

Lie tests (nearly identical questions for consistency).
Use standardised procedures.
Allow anonymity.
Avoid leading questions.

Question 16

What is meant by the term reliability?

Answer 16

Reliability refers to the ability to repeat a study in similar conditions to gain consistent results.

Question 17

Is reliability high or low in a lab experiment and why?

Answer 17

High reliability – control environment – control over extraneous variables.

Question 18

Is reliability high or low in a field experiment and why?

Answer 18

Low reliability – real life environment – low control over extraneous variables.

Question 19

Is reliability high or low in a quasi experiment and why?

Answer 19

Low reliability – naturally occurring IV – low control over extraneous variables

Question 20

: Is reliability high or low in a natural experiment and why?

Answer 20

Low reliability – naturally occurring IV – low control over extraneous variables.

Question 21

How is test-retest used to assess reliability?

Answer 21

• Participants complete a task or measure (CONTEXT).
• After a time delay (e.g. 2 weeks), same task is repeated.
• Results from both tests are correlated.
• A strong positive correlation above +0.8 shows high reliability.

Question 22

What does operationalising mean?

Answer 22

Being specific and clear when defining the IV and DV in an experiment so they can be easily measured.

Question 23

Why is operationalising variables important for reliability?

Answer 23

: If variables are clear and specific, another researcher can repeat the study to check for consistent results (replicability). If consistent, the research is reliable.

Question 24

What is inter-observer reliability?

Answer 24

Two observers are trained on behaviour categories (CONTEXT).
• They watch the same behaviour for the same time, but record independently.
• Tallies are compared and correlated using stats test.
• A strong positive correlation of +0.8 shows high reliability.

Question 25

How can reliability of observations be improved?

Answer 25

• Operationalise behaviour categories. • Use two observers to check consistency. • Train observers on using categories. • Film observations so observers can practise categorising accurately.

Question 26

Q: Is reliability high or low in questionnaires and why?

Answer 26

High reliability – can be repeated to check for consistent results.

Question 27

Is reliability high or low in structured interviews and why?

Answer 27

High reliability – can be repeated to check for consistent results.

Question 28

Is reliability high or low in unstructured interviews and why?

Answer 28

Low reliability – participants get different questions – difficult to repeat.

Question 29

How is test-retest used for questionnaires or interviews?

Answer 29

• Participants complete the questionnaire/interview (CONTEXT). • After a delay (e.g. 2 weeks), same questions given again. • Data is correlated (e.g. scattergraph and stats test). • Strong positive correlation above +0.8 shows high reliability

Question 30

How can questionnaires be made more reliable?

Answer 30

Make sure all questions are clear and understandable – use a pilot study to check.

Question 31

How can interviews be made more reliable?

Answer 31

Use set (structured) questions.

Question 32

Why is operationalising questions in self-reports important?

Answer 32

If questions are vague, the study can’t be repeated. Operationalised questions allow researchers to repeat the study to check for consistent results.