Chapter 6: validity

Question 1

a judgement or estimate of how well a test measures what it purports to measure in a particular context

Answer 1

validity

Question 2

The process of gathering and evaluating evidence about
validity

Answer 2

validation

Question 3

T or F: Both test developers and test users may play a role in the validation of a test

Answer 3

true

Question 4

May yield insights regarding a particular population of test takers as compared to the norming sample described in a test manual

Answer 4

Local validation studies

Question 5

three categories of validity

Answer 5

• content validity
• criterion-related validity
• construct validity

Question 6

This measure of validity is based on an evaluation of the subjects, topics, or content covered by the items in the test

Answer 6

content validity

Question 7

This measure of validity is obtained by evaluating the relationship of scores obtained on the test to scores on other tests or measures

Answer 7

criterion-related validity

Question 8

This measure of validity is arrived at by executing a comprehensive analysis of:
- how scores on the test relate to other test scores and measures
- how scores on the test can be understood within some theoretical framework for
understanding the construct that the test was designed to measure

Answer 8

construct validity

Question 9

A judgment concerning how relevant the test items appear to be

Answer 9

face validity

Question 10

If a test appears to measure what it purports to measure “on the face of it,” it could be said to be __________

Answer 10

high in face validity

Question 11

A judgment of how adequately a test samples behavior representative of the universe of behavior that the test was designed to sample

Answer 11

content validity

Question 12

A plan regarding the types of information to be covered by
the items, the number of items tapping each area of coverage, the organization of the items in the test, etc.

Answer 12

test blueprint

Question 13

Culture and the relativity of content validity

Answer 13

• the content validity of a test varies across cultures and time
• political considerations may also play a role

Question 14

measures
agreement among raters regarding how essential an individual test item is for inclusion in a test

Answer 14

content validity ratio

Question 15

values range of content validity ratio

Answer 15

-1 to 1

Question 16

what does it mean if content validity ratio is closer to +1

Answer 16

majority of experts agree there is an association between the item and the domain

Question 17

the standard against which a test or a test score is evaluated

Answer 17

criterion

Question 18

characteristic of an adequate criterion

Answer 18

• relevant to the matter at hand
• valid for the purpose for which it is being used

Question 19

A judgment of how adequately a test score can be used to infer an individual’s most probable standing on some measure of interest

a. criterion-related validity
b. concurrent validity
c. predictive validity

Answer 19

a. criterion-related validity

Question 20

An index of the degree to which a test score is related to some criterion measure obtained at the same time

a. criterion-related validity
b. concurrent validity
c. predictive validity

Answer 20

b. concurrent validity

Question 21

An index of the degree to which a test score predicts some criterion, measure

a. criterion-related validity
b. concurrent validity
c. predictive validity

Answer 21

c. predictive validity

Question 22

Statistical evidences for concurrent and predictive validity

Answer 22

• expectancy data ~ expectancy table/chart
• validity coefficient

Question 23

A correlation coefficient that provides a measure of the relationship between test scores and scores on the criterion measure

Answer 23

validity coefficient

Question 24

The degree to which an additional predictor explains something about the criterion measure that is not explained by predictors already in use

Answer 24

incremental validity

Question 25

Proportion of people a test accurately identifies a possessing/exhibiting a particular trait, behavior/ characteristic/attribute

Answer 25

hit rate

Question 26

Proportion of people the test fails to identify as having/ not having a particular characteristic/ attribute

Answer 26

miss rate

Question 27

percentage of people hired under the existing system for a particular position extent to which a particular trait, behavior, characteristic or attribute exists in the population expressed in proportion

Answer 27

base rate

Question 28

numerical value that reflects the relationship between the number of people to be hired and the number of people available to be hired

Answer 28

selection ration

Question 29

a miss wherein the test predicted that the examinee did possess the particular characteristic being measured when the examinee did not

Answer 29

false positive (type 1 error)

Question 30

a miss wherein the test predicted that the examinee did not possess the particular characteristic being measured when the examinee did

Answer 30

false negative (type 2 error)

Question 31

Judgment about the appropriateness of inferences drawn from test scores regarding individual standings on a construct

Answer 31

construct validity

Question 32

T or F: If a test is a valid measure of a construct, then high scorers and low scorers should behave as theorized

Answer 32

true

Question 33

evidence of construct validity

Answer 33

- homogeneity - changes with age - pretest-posttest changes - from distinct groups

Question 34

evidence of construct validity: How uniform a test is in measuring a single concept

Answer 34

evidence of homogeneity

Question 35

evidence of construct validity: Some constructs are expected to change over time (e.g., reading rate)

Answer 35

evidence of changes with age

Question 36

evidence of construct validity: Test scores change as a result of some experience between a pretest and a posttest (e.g., therapy)

Answer 36

evidence of pretest-posttest changes

Question 37

evidence of construct validity: Scores on a test vary in a predictable way as a function of membership in some group

Answer 37

evidence form distinct groups

Question 38

Scores on the test undergoing construct validation tend to correlate highly in the predicted direction with scores on older, more established tests designed to measure the same (or a similar) construct

Answer 38

convergent evidence

Question 39

Validity coefficient showing little relationship between test scores and/or other variables with which scores on the test should not theoretically be correlated

Answer 39

discriminant/divergent evidence

Question 40

Class of mathematical procedures designed to identify specific variables on which people may differ

Answer 40

factor analysis

Question 41

A factor inherent in a test that systematically prevents accurate, impartial measurement

Answer 41

bias

Question 42

A judgment resulting from the intentional or unintentional misuse of a rating scale

Answer 42

rating error

Question 43

Raters may be either too lenient, too severe, or reluctant to give ratings at the extremes

Answer 43

central tendency error

Question 44

A tendency to give a particular person a higher rating than he or she objectively deserves because of a favorable overall impression

Answer 44

halo effect

Question 45

The extent to which a test is used in an impartial, just, and equitable way

Answer 45

fairness