Chapter 5

Question 1

a proportion that indicates the ratio between the true score variance on a test and the total
variance

Answer 1

reliability coefficient

Question 2

A statistic useful in describing sources of test score variability is the _______

Answer 2

variance

Question 3

Variance from true differences is _______

Answer 3

true variance

Question 4

variance from irrelevant, random sources
is ________________

Answer 4

error variance

Question 4

refers to the
proportion of the total variance attributed to true variance.

Answer 4

reliability

Question 5

refers to collectively all of the factors associated
with the process of measuring some variable, other than the variable being measured.

Answer 5

measurement error

Question 5

is a source of error in measuring a targeted variable caused by
unpredictable fluctuations and inconsistencies of other variables in the measurement process.

Answer 5

Random error

Question 6

refers to a source of error in measuring a
variable that is typically constant or proportionate to what is presumed to be the true value of
the variable being measured.

Answer 6

systematic error

Question 7

terms that refer to variation among items within a test as well as to
variation among items between tests

Answer 7

item sampling or
content sampling

Question 8

is an estimate of reliability obtained by correlating pairs of scores
from the same people on two different administrations of the same test

Answer 8

Test-retest reliability

Question 9

the estimate of test-retest reliability is often referred to as the ______

Answer 9

coefficient of
stability.

Question 10

The degree of the relationship between various forms of a test can be evaluated by means of an alternate-forms or parallel-forms
coefficient of reliability, which is often termed the ________

Answer 10

coefficient of equivalence.

Question 11

refers to an estimate of the extent to which item sampling and other errors have affected test scores on versions of the same test when, for each form of the test, the means and variances of observed test scores are equal.

Answer 11

parallel forms
reliability

Question 12

are simply different versions of a test that
have been constructed so as to be parallel.

Answer 12

Alternate forms

Question 13

refers to an estimate of the extent to which these different forms of the same test have been affected by item sampling error, or other error.

Answer 13

alternate forms reliability

Question 14

is obtained by correlating two pairs of scores obtained
from equivalent halves of a single test administered once.

Answer 14

split-half reliability

Question 14

This method yields an estimate of split-half
reliability that is also referred to as _________

Answer 14

odd-even reliability.

Question 15

refers to the degree of correlation among all the
items on a scale.

Answer 15

Inter-item consistency

Question 15

allows a test developer or user to estimate internal consistency reliability from a correlation of two halves of a test.

Answer 15

Spearman–Brown formula

Question 16

is the degree to which
a test measures a single factor.

Answer 16

homogeneity

Question 17

may be thought of as the mean of all possible split-half correlations, corrected by the Spearman–Brown formula.

Answer 17

coefficient alpha

Question 18

describes the degree to which a test
measures different factors.

Answer 18

heterogeneity

Question 19

as a measure used to evaluate the internal consistency of a test that focuses on the degree of difference that exists between item scores.

Answer 19

average proportional distance method
(APD)

Question 20

Homogeneity VS heterogeneity of test items (essay)

Answer 20

Recall that a test is said to be homogeneous
in items if it is functionally uniform throughout. Tests designed to measure one factor, such as one ability or one trait, are expected to be homogeneous in items. For such tests, it is reasonable to expect a high degree of internal consistency. By contrast, if the test is heterogeneous in items, an estimate of internal consistency might be low relative to a more appropriate estimate of test-retest reliability.

Question 21

is the degree of agreement or consistency between two or more scorers (or judges or raters) with regard to a particular measure.

Answer 21

inter-scorer reliability

Question 22

is a trait, state, or ability presumed to be ever-changing as a function of situational and cognitive experiences.

Answer 22

dynamic characteristic

Question 23

ability presumed to be relatively unchanging is ______ such as intelligence.

Answer 23

static characteristic

Question 24

if some items are so difficult that no test-taker is able to obtain a perfect score, then the test is a _____

Answer 24

power test

Question 25

generally contains items of uniform level of difficulty (typically uniformly low) so that, when given generous time limits, all test-takers should be able to complete all the test items correctly

Answer 25

speed test

Question 25

is designed to provide an indication of where a test-taker stands with respect to some variable or criterion, such as an educational or a vocational objective

Answer 25

criterion-referenced test

Question 26

a value that according to classical test theory genuinely reflects an individual’s ability (or trait) level as measured by a particular test.

Answer 26

true score

Question 26

also referred to as the true score (or classical) model of measurement. _________ is the most widely used and accepted model in the psychometric literature today

Answer 26

classical test theory (CTT)

Question 27

seek to estimate the extent to which specific sources of variation under defined conditions are contributing to the test score.

Answer 27

domain sampling theory

Question 28

is based on the idea that a person’s test scores vary from testing to testing because of variables in the testing situation.

Answer 28

generalizability theory

Question 29

Cronbach encouraged test developers and researchers to describe the details of the particular test situation or ______ leading to a specific test score.

Answer 29

Universe

Question 30

examines how generalizable scores from a particular test are if the test is administered in different situations.

Answer 30

generalizability study

Question 30

include things like the number of items in the test, the amount of training the test scorers have had, and the purpose of the test administration.

Answer 30

Facets

Question 31

These coefficients are similar to reliability coefficients in the true score model.

Answer 31

coefficients of generalizability.

Question 32

developers examine the usefulness of test scores in helping the test user make decisions.

Answer 32

decision study

Question 33

Another alternative to the true score model is ________

Answer 33

Item response theory (IRT)

Question 34

a synonym for Item response theory (IRT) in the academic literature is _____

Answer 34

latent-trait theory.

Question 35

is a categorical variable with two possible response values (Yes/No, Agree/Disagree, Success/Fail).

Answer 35

dichotomous item

Question 36

is a categorical variable ordinal or nominal with more than two possible values (e.g. strongly disagree, disagree, agree, strongly agree).

Answer 36

polytomous item

Question 37

is a reference to an IRT model with very specific assumptions about the underlying distribution

Answer 37

Rasch model

Question 38

is the tool used to estimate or infer the extent to which an observed score deviates from a true score.

Answer 38

standard error of measurement

Question 39

a range or band of test scores that is likely to contain the true score.

Answer 39

confidence interval

Question 40

Comparisons between scores are made using the _________

Answer 40

standard error of the difference

Question 41

refers to a group of personality tests.

Answer 41

Personality test battery