Week 6: Reliability

Question 1

validity: design vs measurement

Answer 1

[statistical conclusion validity, internal validity, external validity] =/= [construct validity, measurement reliability]

Question 2

6 indicators of construct validity

Answer 2

• face validity
• content validity
• predictive validity
• concurrent validity
• convergent validity
• discriminative validity

Question 3

face validity

Answer 3

content of measure appears to reflect construct being measured (eg rating scale)
- based on judgement of instrument

Question 4

content validity

Answer 4

• comparing content of measurement w content that defines construct
• questionnaire w good content validity should include questions on all components

Question 5

predictive validity

Answer 5

• measurement to predict future behaviour/outcome

Question 6

concurrent validity

Answer 6

• relationship between measurement and criteria at the same time (concurrently)
• important when trying to validate new instrument
• demonstrated when test correlates well w measure previously validated

Question 7

convergent validity

Answer 7

measuring 2 theoretical constructs that are related in theory -> measurements should also be related
- extent to which scores are related to other measures

Question 8

discriminative validity

Answer 8

• degree to which measurement is NOT related to other (not theoretically related) measurements

Question 9

threats to construct validity

Answer 9

• inadequate description of construct
• inadequate measurement
• inadequate attention to IV levels

Question 10

inadequate description of construct

Answer 10

• focus on limited number of components -> concluding that entire construct is evaluated

Question 11

inadequate measurement

Answer 11

only one measurement/method to gather data

Question 12

inadequate attention to IV levels

Answer 12

only one/two levels of multilevel variable are used

Question 13

reliability

Answer 13

• replicability/consistency

Question 14

internal reliability

Answer 14

• reliability of instrument chosen to evaluate DV in study

Question 15

reliability coefficient

Answer 15

#; relationship between multiple test administration/multiple items 
- 0 - 1 scale (0.8 is good)

Question 16

reliability of instrument (internal reliability)

Answer 16

evaluated by:

• parallel forms
• test-retest
• split-half
• Cronbach’s alpha

Question 17

parallel forms reliability

Answer 17

• similarity between different sets of questions targeting same construct
• different test to different people

Question 18

test-retest reliability

Answer 18

• over time
• testing same people on same test on different times
• expected to yield similar results

Question 19

split-half reliability

Answer 19

• expected to score similar on both halves

- both halves used in same test

Question 20

Cronbach’s alpha

Answer 20

• estimate of different items on same scale to represent same content
• calculated between each pair of questions on test

Question 21

reliability of observations (2)

Answer 21

• inter-rated reliability

- intra-rater reliability

Question 22

inter-rated reliability

Answer 22

• degree to which 2 independent raters/observers record/code same situation similarly

Question 23

intra-rater reliability

Answer 23

• degree to which same rater/observer records similar data about same observation on 2 different occasions