CHAPTER 5 RELIABILITY

Question 1

this index of reliability is a proportion that indicates the ratio between the true score variance on a test and the total variance.

Answer 1

Reliability Coefficient

Question 2

this statistic is useful in describing sources of test score variability (the standard deviation squared)

Answer 2

Variance

Question 3

what is true variance?

Answer 3

variance from true differences

Question 4

this term refers to the proportion of the total variance attributed to true variance

Answer 4

Reliability

Question 5

what is error variance?

Answer 5

variance from irrelevant, random
sources

Question 5

this source of error comes from unpredictable fluctuations and inconsistencies of other variables in the measurement process

Answer 5

Random Error

Question 5

what are the two categories of measurement error?

Answer 5

random error and systematic error

Question 5

this type of variable involves pressing emotional problems, physical discomfort, lack of sleep, and the effects of drugs or
medication

Answer 5

testtaker variables

Question 6

this error refers to all of the factors associated with the process of measuring some variable, other than the variable being measured.

Answer 6

Measurement error

Question 7

this source of error is typically constant or proportionate to what is presumed to be the true value of the variable being measured

Answer 7

Systematic Error

Question 8

what are the sources of error variance?

Answer 8

1. Test Construction
2. Test Administration
3. Test Scoring and Interpretation

Question 9

this term refers to the practice of administering different subsets of items to different test takers

refers to the variation of items both within a single test and between different tests

Answer 9

Item Sampling (Content Sampling)

Question 10

this type of variables involve the examiner’s physical appearance and demeanor—even the presence or absence of an examiner—are some factors for consideration here

Answer 10

Examiner-related Variables

Question 11

for test-retest reliability, what is its source of error, procedure, reliability coefficient/index, and when is it used?

Answer 11

source of error: time

procedure: correlating the same test at different times

reliability coefficient/index: coefficient of stability

used for time-dependent variables

Question 12

this term is an estimate of reliability obtained by correlating pairs of scores from the same people on two different administrations of the same test.

Answer 12

Test-Retest Reliability Estimates

Question 13

what are the types of reliability estimates?

Answer 13

1. Test-Retest
2. Parallel-Forms & Alternate Forms
3. Split-Half Reliability
4. Internal Consistency Reliability

Question 14

what are the estimates used in internal consistency reliability?

Answer 14

1. KR20
2. KR21
3. Cronbach’s Alpha

Question 15

this reliability estimate refers to an estimate of the extent to which item sampling and other errors have affected test scores on versions of the same test when, for each form of the test, the means and variances of observed test scores are equal.

Answer 15

Parallel-Forms Reliability

Question 16

this reliability estimate refers to an estimate of the extent to which these different forms of the same test have been affected by item sampling error, or other error

Answer 16

Alternate Forms of Reliability

Question 17

for parallel forms and alternate-forms reliability, what is its source of error, procedure, reliability coefficient/index, and when is it appropriate

Answer 17

source of error: item form/structure/arrangement

procedure: correlate the scores with two tests, with the same everything including test takers

reliability coefficient/index: coefficient of equivalence

appropriate for form or arrangement dependent variables

Question 18

this reliability estimate is obtained by correlating two pairs of scores obtained from equivalent halves of a single test
administered once.

Answer 18

Split-Half Reliability

Question 19

for internal consistency reliability, what is its source of error, procedure, reliability coefficient/index, and when is it appropriate

Answer 19

source of error: item heterogeneity

procedure: Kuder-Richardson 20/21 forms

reliability coefficient/index: coefficient of item homogeneity

appropriate for test that are dichotomous or binary response

Question 20

when is KR20 appropriate?

Answer 20

when it is a true dichotomous test (true or false)

Question 21

when is KR21 appropriate?

Answer 21

when it is an artificial dichotomy (multiple choice, binary scoring)

Question 22

when is Cronbach's alpha appropriate?

Answer 22

when it is a true multiple choice test (Likert Scale)

Question 22

what are the estimates categorized as external reliability estimates? and why are they called external?

Answer 22

test-retest parallel-forms and alternate forms reliability called external because they assess the consistency of a measure across different individuals or administrations

Question 22

what are the estimates categorized as internal reliability estimates?

Answer 22

split-half internal consistency

Question 23

what should the value for cronbach's alpha always be?

Answer 23

a value between 0 and 1 the closer to 1, the more homogeneous it is

Question 24

what is the threshold for cronbach's alpha?

Answer 24

0.70