Exam 2

Question 1

consistency in measurement of some (real or hypothetical) characteristic

Answer 1

reliability

Question 2

the theoretical number that it one’s perfectly accurate representation of knowledge as a score on a test

Answer 2

True Score

Question 3

a person’s true score, plus some error

Answer 3

observed test score

Question 4

unsystematic variability introduced into scores; random; can cause deviation from true score

Answer 4

error

Question 5

Sources of Psychological Measurement Error

Answer 5

• test construction
• test administration
• test scoring

Question 6

items are created/selected from a large population of possible items from within the domain
may affect how well one performs on a test. Someone may perform very well on some subtests and bad on others.

Answer 6

content sampling

Question 7

factors that influence the test taker’s attention, concentration, motivation, etc.

Answer 7

test administration

Question 8

physical appearance, departure from test standardization procedure, not placing materials in proper orientation, incorrect timing, etc.

Answer 8

examiner influences

Question 9

test environment, test anxiety, medication effects, extended testing session - fatigue

Answer 9

physical or psychological discomfort

Question 10

estimates of the ratio of true score variance to total variance (cannot be negative)

Answer 10

reliability coefficients

Question 11

test-retest/stability, interrater agreement. internal consistency, alternate forms, etc.

Answer 11

types of reliability

Question 12

reliability indices should meet or exceed

Answer 12

.85 or .90

Question 13

index of how an individual’s scores may vary over tests presumed to be parallel

Answer 13

standard error of measurement (SEm)

Question 14

as rxx increases from 0-1…

Answer 14

SEm decreases from SD to 0

Question 15

Purpose of rxx

Answer 15

1. internal consistency
2. alternate forms
3. test-retest
4. interrater agreement

Question 16

static vs. dynamic characteristic

Answer 16

static doesn’t change very much

dynamic evolves a lot

Question 17

degree to which evidence and theory support the interpretations of test scores for proposed uses of tests
relates to inferences or interpretations made about performance based on scores from the measure.

Answer 17

validity

Question 18

• content validity
• criterion related validity
• construct validity

Answer 18

trinitarian model

Question 19

Evidence based on:

• test content
• response processes
• internal structure
• relations with other variables
• consequences of testing

Answer 19

unified theory (Messick)

Question 20

examination of test content

Answer 20

content validity

Question 21

• test-retest
• alternate forms
• interrater agreement

Answer 21

Methods of Reliability

Question 22

Rα

Answer 22

Cronbach’s coefficient alpha

Question 23

when an item belongs to more than one factor

Answer 23

cross loading

Question 24

examination of test content

Answer 24

content validity

Question 25

• test-retest
• alternate forms
• interrater agreement

Answer 25

Methods of Reliability

Question 26

Rα

Answer 26

Cronbach’s coefficient alpha

Question 27

when an item belongs to more than one factor

Answer 27

cross loading

Question 28

z scores for 68%
95%
99%

Answer 28

1

1. 96
2. 58

Question 29

Which reliability estimate should be used with test-retest?

Answer 29

stability

Question 30

SEdiff formula

Answer 30

SEdiff = SD√2-rxx1-rxx2
Rxx1 = reliability of test 1
Rxx2 = reliability of test 2
answer is the number of points required to make a SIGNIFICANT DIFFERENCE

Question 31

Which reliability estimate should be used with internal consistency?

Answer 31

coefficient alpha, KR-20, split-half

Question 32

Which reliability estimate should be used with alternate forms?

Answer 32

equivalance

Question 33

Which reliability estimate should be used with test-retest?

Answer 33

stability

Question 34

compare a test in question with an already accepted standard measure

Answer 34

concurrent validity

Question 35

when correlations are superficially high because two tests are too similar

Answer 35

criterion contamination

Question 36

test in question is compared to a standard gathered at a future time; predicts performance on a standard at a future time
ex: hs GPA and ACT predicting college GPA

Answer 36

predictive validity

Question 37

experimental procedures that cause a change in construct can be used to test a test’s validity

Answer 37

theory consistent intervention effects

Question 38

two or more measures designed to measure the same construct should produce high correlations (large amount of shared variance)

Answer 38

convergent validity

Question 39

two or more measures designed to measure different constructs should produce low/near zero correlations (little shared variance)

Answer 39

divergent validity

Question 40

use scores to classify subjects into groups

Answer 40

discriminant analysis

Question 41

conceptualization

Answer 41

is there a need? who wil it be used for? content? administration? responses?

Question 42

scaling

Answer 42

how we assign “points” to responses

Question 43

try-out, analyze, revise

Answer 43

first “test” administration, analysis, revision

Question 44

Spearman brown

Answer 44

Used to estimate reliability after changing test length

Question 45

KR-20

Answer 45

Used with dichotomous