Reliability & Validity

Question 1

What is reliability?

Answer 1

A measure of consistency

Question 2

What is internal reliability?

Answer 2

The consistency within a test

Question 3

What is external reliability?

Answer 3

The ability to produce the same results every time the test is carried out

Question 4

How can you improve reliability in an experiment?

Answer 4

standardisation

Take more than one measure

Pilot studies

Question 5

What is standardisation?

Answer 5

Making sure the procedures are the same each time for each participant

Question 6

Why should you take more than one measure?

Answer 6

reduces the impact of an anomalous score by repeating the measurement

Question 7

What is a pilot study?

Answer 7

Small scale study done to discover any problems with the design

Question 8

How can you assess reliability in an experiment?

Answer 8

Test retest reliability

Question 9

What is test retest reliability?

Answer 9

• The test/procedure is used several times with the same group of participants
• Participant score from the first test should be correlated with the second test
• Should be a short interval between each tests

Question 10

How can you improve a reliability in an observation?

Answer 10

behavioural categories

Pilot studies

Standardisation

Question 11

How can behavioural categories improve reliability?

Answer 11

Must be operationalised

Question 12

how can standardisation improve reliability in an observation?

Answer 12

when more than one investigator is used the way they collect data to be standardised

Question 13

How can you assess reliability in an observation?

Answer 13

inter observer reliability

Question 14

what is inter observer reliability?

Answer 14

Two observers decide on a set of categories

they each carry out the observation independently and tally each behavioural category

at the end observers compare and correlate their totals

A correlation coefficient of 0.8 or more suggests good into observe reliability

Question 15

how can you improve the reliability of self report?

Answer 15

reduce ambiguity

Pilot studies

Standardisation

Question 16

How can you reduce ambiguity of a self-report?

Answer 16

By reducing the chance of a question being misunderstood
- changing questions to closed questions

Question 17

How can you access reliability on a self report?

Answer 17

test retest reliability

Repeat the question/interview

Question 18

How can you assess reliability of the split half method?

Answer 18

Each participant scores on one half of the test should be correlated with their scores on the other half

reliability is then determined by comparing the calculated coefficient with the critical value s

Question 19

What is validity?

Answer 19

I measure of accuracy
Does something measure what it’s supposed to measure?

Question 20

What is internal validity?

Answer 20

The ability of the study to test the hypothesis it was designed to test

Question 21

How can you decide if a study is valid?

Answer 21

variable such behaviour category should be operationalised

Controlling confounding variables

The experimental design

Question 22

what are some confounding variables that may affect validity?

Answer 22

Demand characteristics

Research bias

Participate variables

Question 23

What is external validity?

Answer 23

How well results of the three can be generalised beyond the study

Question 24

what is ecological validity?

Answer 24

The extent which research can be generalised to situations other than the research setting

E.g. can research in a lab generalised to real life

Question 25

what is population validity?

Answer 25

Weather results from the limited sample can be generalised to the target population

Question 26

what is historical validity?

Answer 26

If results can be generalised to another historical period

Question 27

How can you assess validity?

Answer 27

Face validity Concurrent validity Content validity

Question 28

what is face validity?

Answer 28

A common sense judgement about a test validity

Question 29

what is concurrent validity?

Answer 29

If the test produces similar findings to another existing measure

Question 30

How can you assess concurrent validity?

Answer 30

same participants complete two tests assessing the same aim Scores from one test should be collected with the other test

Question 31

What is content validity?

Answer 31

assessed by asking experts in the field to check the methodology of a study see how accurately it measures desired behaviour

Question 32

how can removing questions improve validity?

Answer 32

If it has poor face/concurrent validity questions should be revised so that they relate more to the topic

Question 33

how can improving the research design improve validity?

Answer 33

it can control: investigator effects demand characteristics Confounding variables unoperationalised behaviour categories

Question 34

How can you improve ecological validity

Answer 34

give ppts a task with high mundane realism conduct a field/natural experiment

Question 35

How can you improve the population validity

Answer 35

use a larger sample use a sampling technique thats likely to produce a representative sample

Question 36

How does single blind improve validity

Answer 36

reduced demand characteristics as ppts dont know what group or condition theyre in

Question 37

How does double blind improve validity

Answer 37

reduces demand characteristics AND researcher bias neither ppts or the researcher know what what each condition represents