Validity

Question 1

It is a judgment based on evidence about the appropriateness of inferences drawn from test scores

Answer 1

Validity

Question 2

Validation process if test users plan to alter format, instruction, language or content of the test

Answer 2

Local validation

Question 3

Trinatarian view

Answer 3

Content, criterion, construct validity

Question 4

Judgment concerning how relevant a test items appear to be

Answer 4

Face validity

Question 5

How adequately a test samples behavior representative of the universe of behavior that the test was designed to sample; coverage, course syllabus, curricula

Answer 5

Content validity

Question 6

A plan regarding the types of info to be covered by the items the numbers of items tapping each area of coverage, organization of items and so forth

Answer 6

Test blueprint

Question 7

Manage creation of new items and the output of old items in a manner that is consistent with the test’s blueprint

Answer 7

item pool management

Question 8

In quantifying content validity, this is used to gauge agreement among raters or judges regarding how essential a particular item is

Answer 8

Lawshe content validity ratio (CVR)

Question 9

Negative=fewer than half the panelists indicate item is essential
Zero= exactly half the panelists indicate essential
Positive=more than half but not all panelists indicate essential item

Answer 9

Content Validity ratio

Question 10

Judgment of how adequately a test core can be used to infer an individual’s most probable standing on some measure of interest

Answer 10

Criterion-related validity

Question 11

A criterion must be ___, ___, and ___.

Answer 11

relevant, valid and uncontaminated

Question 12

Happens when criterion overlap with predictor measures

Answer 12

Criterion contamination

Question 13

Test scores are obtained about the same time criterion measures are obtained; also used to compare tests with different versions

Answer 13

Concurrent validity

Question 14

How accurately scores on a test predict some criterion measure

Answer 14

predictive validity

Question 15

Correlation coefficient that provides a measure of the relationship between test scores and scores on the criterion measure

Answer 15

Validity coefficient

Question 16

Degree to which an additional predictor explains something about the criterion measure that is not explained by predictors already in use

Answer 16

Incremental validity

Question 17

Provide info that can be used in evaluating criterion-related validity of a test

Answer 17

Expectancy data

Question 18

Likelihood that testtaker will score within somwe interval of scores on a criterion measure (an interval that may be seen as “passing or acceptable”

Answer 18

Expectancy table

Question 19

Provide an estimate of the percentage of employess by the use of a particular test who will be successful at their jobs.

Answer 19

Taylor-russel tables (comprises test’s validity, selection ratio, base rate)

Question 20

Numerical value that reflects relationship between number of people to be hired and number of people available to be hired

Answer 20

Selection ratio

Question 21

Percentage of people hired under the existing system for a particular position

Answer 21

Base rate

Question 22

Limitations of the Taylor-Russel table

Answer 22

• cannot distinguish differences between successful from unsuccessful employees
• relationship between predictor and criterion must be linear

Question 23

Obtain difference between the means of the selected and unselected groups to derive an index of what the test is adding to the already established procedures

Answer 23

Naylor-Shine table

Question 24

Provide an estimate of the extent to which inclusion of a particular test in the selection system will actually improve selection

Answer 24

Taylor-Russel tables

Question 25

Theory that explains the usefulness or practical value of tests

Answer 25

Test utility theory

Question 26

Statistical rules for developing a sequential analysis of a problem that would lead to an optimal decision

Answer 26

Decision theory

Question 27

Extent to which a particular trait or behavior exists in the population

Answer 27

Base rate

Question 28

Proportion of people a test accurately idetifies as possessing or exhibiting a particular trait, behavior, characteristic

Answer 28

Hit rate

Question 29

Proportion of people the test fails to identify as having or not having a particular characteristic or attribute

Answer 29

Miss rate

Question 30

Employers are reluctant to use this theory because of the complexity of the application and the threat of legal challenges

Answer 30

Decision theory

Question 31

One measure of the value of a test lies in the extent to which its use improves on the hit rate that exists without using ut

Answer 31

To remember

Question 32

If a test is a valid measure of the construct, then high scorers and low scorers will behave as predicted by the theory

Answer 32

Construct validity

Question 33

Prove validity by demonstrating that scores on the test vary in a predictable was as a function of membership in some group

Answer 33

Method of contrasted groups

Question 34

Proof that a test provides scores similar to an older, established, and valid test of the same construct

Answer 34

Convergent evidence

Question 35

No significant or little relationship between test scores and other variables should not be theoretically related to the construct

Answer 35

Discriminant evidence

Question 36

Examine both convergent and discrimanant validity

Answer 36

Multitrait-multimethod matrix

Question 37

Frequently employed as a data reduction method

Answer 37

factor analysis

Question 38

Estimating number of factors, deciding how many to maintain

Answer 38

Exploratory factor analysis

Question 39

Tested for its fit with the observed covariance structure of the measured variables

Answer 39

Confirmatory FA

Question 40

The extent to which the factor determines the test scores

Answer 40

factor loading (low: provide discrimanant evidence; high: provide convergent evidence)

Question 41

Inherent factor in a test that systematically prevents accurate measurement

Answer 41

Bias

Question 42

When a test systematically under-predicts or over-predicts the performance of a particular group

Answer 42

intercept bias (regression line intercepts y-axis)

Question 43

When the slope of one group's regression line differs significantly from another's

Answer 43

slope bias | solution: estimated true score transformations

Question 44

Numerical or verbal judgment that places a person along a continuum

Answer 44

rating

Question 45

Judgment resulting from intentional or unintentional misuse of a rating scale

Answer 45

RATING ERROR

Question 46

How to overcome the 3 rating errors (leniency, severity, central tendency)

Answer 46

Thru RANKINGS (measure individuals against one another insetad of against an absolute scale)

Question 47

Tendency to give a particular ratee a higher rating than he or she objectively deserves beacuse of the rater;s failure to discriminate among conceptually distinct and potentially independent aspects of ratee's behavior; the ratee "can do no wrong"

Answer 47

Halo effect

Question 48

Extent to which a test is used in an impartial, just, and equitable way; more rooted in issues involving values

Answer 48

fairness