CH5 - RELIABILITY

Question 1

what is reliability?

Answer 1

consistency in measurement

Question 2

this term refers to a statistic statistic that quantifies reliability, ranging from 0 (not at all reliable) to 1 (perfectly reliable)

Answer 2

reliability coefficient

Question 3

what is measurement error?

Answer 3

measurement error refers to the inherent uncertainty associated with any measurement, even after care has been taken to minimize preventable mistakes

Question 4

does the true score necessarily reflect the truth?

Answer 4

no, for example a person’s score on a depression questionnaire would differ from their true score on another measurement since depression questionnaires emphasize different aspects of depression

Question 5

how do we formulize the concept of the observed score?

X (observed score)
T (true score)
E (measurement error)

Answer 5

X = T+E

Question 6

what does it mean when people’s
observed scores are mostly determined by their true scores?

Answer 6

the test is reliable

Question 7

what does it mean when people’s
observed scores are mostly determined by measurement error?

Answer 7

the test is unreliable

Question 8

what statistic is useful in describing sources of test score variability?

Answer 8

variance (σ2)—the standard deviation squared.

Question 9

what is true variance?

Answer 9

variance from true differences

Question 10

what is error variance?

Answer 10

variance from irrelevant, random sources

Question 11

how do we formulize total observed variance?

Answer 11

σ 2 = σ 2t + σ 2e

Question 12

what can reliability also refer to?

Answer 12

the proportion of the total variance attributed to true variance.

Question 13

what is the difference between random errors and systematic errors?

Answer 13

random errors cancel each other out while systematic errors do not because systematic errors influence test scores in a consistent direction

Question 14

what is bias?

Answer 14

bias refers to the degree to which systematic error influences the measurement.

Question 15

how is test construction considered as a source of error variance?

Answer 15

the content sampled in the tests affect a test taker’s score

Question 15

what are the sources of error variance? (there are 4)

Answer 15

1. test construction
2. test administration
3. scoring
4. interpretation

Question 16

what are other sources of error?

Answer 16

1. sampling error
2. methodological error
3. nonsystematic error (forgetting or misunderstanding instructions regarding reporting)

Question 16

how do we estimate the reliability of a measuring instrument?

Answer 16

we use the same instrument to measure the same thing at two points in time. (test-retest method)

Question 17

what is test-rest reliability?

Answer 17

an estimate of reliability using the test-retest method

Question 17

when is the test-retest measure appropriate?

Answer 17

the test-retest measure is appropriate when evaluating the reliability of a test that purports to measure something thatis relatively stable over time, such as a personality trait

Question 19

what is coefficient of stability?

Answer 19

the estimate of test-retest reliability when the interval between testing is greater than six months

Question 20

this term refers to the degree of the relationship between various forms of a test can be evaluated by means of an alternate-forms or parallel-forms coefficient of reliability

Answer 20

coefficient of equivalence

Question 21

what are parallel forms of a test?

Answer 21

parallel forms of a test exist when the means and variances of observed test scores are equal

Question 22

what is parallel forms reliability?

Answer 22

parallel forms reliability refers to an estimate of the extent to which item sampling and other errors have affected test scores on versions of the same test when, for each form of the test, the means and variances of observed test scores are equal.

Question 23

what are alternate forms of a test?

Answer 23

alternate forms are simply different versions of a test that have been constructed so as to be parallel

Question 23

this term refers to an estimate of the extent to which these different forms of the same test have been affected by item sampling error, or other error

Answer 23

alternate forms reliability

Question 23

how do you estimate alternate forms reliability?

Answer 23

calculate the correlation between scores from a representative sample of individuals who have taken both tests

Question 23

this term refers to the evaluation of the internal consistency of the test items

Answer 23

internal consistency estimate of reliability

Question 24

what is split-half reliability?

Answer 24

a method/estimate of obtaining internal consistency estimate of reliability by by correlating two pairs of scores obtained from equivalent halves of a single test administered once

Question 25

how do u compute the coefficient of split-half reliability?

Answer 25

Step 1. Divide the test into equivalent halves. Step 2. Calculate a Pearson r between scores on the two halves of the test. Step 3. Adjust the half-test reliability using the Spearman–Brown formula (discussed shortly).

Question 26

how do you split a test?

Answer 26

1. randomly assign items to one or the other half of the test. 2. assign odd-numbered items to one half of the test and even-numbered items to the other half

Question 27

what is the primary objective in splitting a test for the purpose of obtaining a split-half reliability estimate?

Answer 27

it is to create what can be called as mini-parallel forms, with each half equal to each other, in format and statistical aspects