Test construction

Question 1

Incremental validity

Answer 1

Criterion validity; amount of improvement in success rate from using a predictor test

Is optimized when base rate is moderate and selection ratio is low

Question 2

Concurrent Validity

Answer 2

Criterion validity; predictor and criterion are measured close in time

Question 3

Predictive validity

Answer 3

Criterion validity; Significant gap in time between measuring the predictor and the criterion.

Question 4

Face Validity

Answer 4

Level of test subjectively appears to measure what it’s intended to

Question 5

Consequential validity

Answer 5

Addresses whether a test has a positive or negative social consequence.

Question 6

Convergent validity

Answer 6

Construct validity; correlates a test with other available measures of the same trait

Monotrait-heteromethod; high correlations establish convergent validity

Question 7

Divergent (discriminant) validity

Answer 7

Construct validity; correlates a test with measures of different traits. Should have low correlation with other measures of different constructs

Heterotrait- monomethod; low correlations establish divergent validity

Question 8

Sensitivity

Answer 8

Tests ability to accurately identify true positives

TP divided by TP + FN

Question 9

Specificity

Answer 9

Tests ability to accurately identify true negatives

TN divided by TN + FP

Question 10

3 types of validity and their subtypes

Answer 10

Content, Criterion; concurrent, predictive, Construct; convergent, divergent

Question 11

Reliability coefficient

Answer 11

Rxx; ranges from 0.0 (no reliability) to 1.0 (perfect).

Correlation between two sets of scores x and x.

Minimum acceptable value is 0.8
Never squared; if rxx is 0.85 the true score variability would be 85%

Question 12

Test-Retest

Answer 12

Determines tests consistency over time.
People administered the same test at two different times.
Reliability decreases the longer the time between tests.

Question 13

Alternate forms

Answer 13

Also known as: parallel forms or equivalent forms.

Similar, but not identical versions of a test are administered to the same group at two different times

Question 14

Z-score

Answer 14

Mean of 0 standard deviation of 1.0

Question 15

T-score

Answer 15

Mean of 50 and SD of 10

Question 16

Stanines

Answer 16

Mean of 5 and standard deviation of 2 ranges from 1-9.

Question 17

Conversion of raw scores to percentile ranks

Answer 17

Non linear transformation since percentile rank distribution is always rectangular (flat) regardless of the shape of the distribution raw scores.