Chap 6: EVALUATING SELECTION TECHNIQUES AND DECISIONS

Question 1

The extent to which a score from a test or from an evaluation is consistent and free from error.

Answer 1

Reliability

Question 2

method each one of several people take the same test
twice.

Answer 2

test-retest reliability

Question 3

The scores from the first administration of the test are correlated with scores from the second to determine whether they are similar

Answer 3

test-retest reliability

Question 4

The extent to which repeated administration of the same test will achieve similar results.

Answer 4

test-retest reliability

Question 5

The test scores are stable across time and not highly susceptible to such random daily conditions as illness, fatigue, stress, or uncomfortable testing conditions

Answer 5

temporal stability

Question 6

The consistency of test scores
across time.

Answer 6

temporal stability

Question 7

two forms of the same test are constructed

Answer 7

alternate-forms reliability

Question 8

designed to eliminate any effects that taking one form of the test first may have on scores on the second form.

Answer 8

counterbalancing

Question 9

The extent to which two forms of the same test are similar

Answer 9

alternate-forms reliability

Question 10

A method of controlling for order
effects by giving half of a sample Test A first, followed by Test B, and giving the other half of the sample Test B first, followed by Test A

Answer 10

counterbalancing

Question 11

The extent to which the scores on two forms of a test are similar.

Answer 11

Form stability

Question 12

consistency with which an applicant responds to items measuring a similar dimension or construct

Answer 12

Internal Reliability

Question 13

The extent to which similar items are answered in similar ways is referred to as internal consistency and measures ______

Answer 13

item stability

Question 14

The extent to which similar items are answered in similar ways is referred to as _____ and measures item stability

Answer 14

internal consistency

Question 15

The extent to which test items measure the same construct.

Answer 15

Item homogeneity

Question 16

3 statistics used to determine internal reliability of test

Answer 16

-Kuder-Richardson 20
-Spearman-Brown Prophecy Formula
-Coefficient Alpha (Cronbach’s Alpha)

Question 17

• A form of internal reliability in which the consistency of item responses is determined by comparing scores on half of the items with scores on the other half of the items.
• is the easiest to use, as items on a test are split into two groups.

Answer 17

Split-half method

Question 18

are more popular and accurate methods of determining internal reliability, although they are more complicated to compute

Answer 18

Cronbach’s coefficient alpha and the K-R 20

Question 19

Used to correct reliability coefficients resulting from the split-half method.

Answer 19

Spearman-Brown prophecy formula

Question 20

A statistic used to determine internal reliability of tests that use
interval or ratio scales.

Answer 20

Coefficient alpha

Question 21

A statistic used to determine
internal reliability of tests that use items with dichotomous answers (yes/no, true/false).

Answer 21

Kuder-Richardson Formula 20 (K-R 20)

Question 22

used for tests containing dichotomous items (e.g., yes-no,
true-false)

Answer 22

K-R 20

Question 23

can be used not only for dichotomous items but also for tests containing interval and ratio(nondichotomous) items such as five-point rating scales

Answer 23

coefficient alpha

Question 24

The extent to which two people scoring a test agree on the test score, or the extent to which a test is scored correctly.

Answer 24

Scorer reliability

Question 25

an issue in projective or subjective tests in which there is no one correct answer, but even tests scored with the use of keys suffer from scorer mistakes

Answer 25

Scorer reliability

Question 26

When human judgment of performance is involved, scorer reliability is discussed in terms of

Answer 26

interrater reliability

Question 27

when Evaluating the Reliability of a Test, two factors must be considered:

Answer 27

- the magnitude of the reliability coefficient - the people who will be taking the test.

Question 28

The degree to which inferences from test scores are justified by the evidence.

Answer 28

Validity

Question 29

The extent to which tests or test items sample the content that they are supposed to measure.

Answer 29

Content validity

Question 30

In industry, the appropriate content for a test or test battery is determined by the _______

Answer 30

job analysis

Question 31

The extent to which a test score is related to some measure of job performance.

Answer 31

Criterion validity

Question 32

- a test is given to a group of employees who are already on the job. - A form of criterion validity that correlates test scores with measures of job performance for employees currently working for an organization

Answer 32

concurrent validity

Question 33

- design, the test is administered to a group of job applicants who are going to be hired - A form of criterion validity in which test scores of applicants are compared at a later date with a measure of job performance.

Answer 33

predictive validity

Question 34

Difference between concurrent and predictive validity

Answer 34

Concurrent - already on the job. Predictive - applicants who are going to be hired

Question 35

performance scores makes obtaining a significant validity coefficient more difficult

Answer 35

restricted range

Question 36

the extent to which a test found valid for a job in one location is valid for the same job in a different location

Answer 36

validity generalization (VG)

Question 37

it is the most theoretical of the validity types

Answer 37

Construct validity

Question 38

The extent to which a test actually measures the construct that it purports to measure.

Answer 38

Construct validity

Question 39

is concerned with inferences about test scores

Answer 39

Construct validity

Question 40

is concerned with inferences about test construction.

Answer 40

content validity

Question 41

is usually determined by correlating scores on a test with scores from other tests

Answer 41

Construct validity

Question 42

A form of validity in which test scores from two contrasting groups “known” to differ on a construct are compared.

Answer 42

Known-group validity

Question 43

is the extent to which a test appears to be job related.

Answer 43

Face validity

Question 44

True or False face-valid tests resulted in high levels of test-taking motivation, which in turn resulted in higher levels of test performance

Answer 44

true

Question 45

statements that are so general that they can be true of almost anyone.

Answer 45

Barnum Statements

Question 46

- A book containing information about the reliability and validity of various psychological tests. - which contains information on over 2,700 psychological tests as well as reviews by test experts.

Answer 46

Mental Measurements Yearbook (MMY)

Question 47

what edition of Mental Measurements Yearbook (MMY) is used

Answer 47

19th edition

Question 48

A type of test taken on a computer in which the computer adapts the difficulty level of questions asked to the test taker’s success in answering previous questions

Answer 48

Computer-adaptive testing (CAT)

Question 49

designed to estimate the percentage of future employees who will be successful on the job if an organization uses a particular test

Answer 49

Taylor-Russell tables

Question 50

Three information needed by Taylor-Russell Tables

Answer 50

- criterion validity coefficient - Selection ratio (The percentage of applicants an organization hires. hired over applicants) - Base rate (Percentage of current employees who are considered successful.)

Question 51

- A utility method that compares the percentage of times a selection decision was accurate with the percentage of successful employees. - easier to do but less accurate than the Taylor-Russell tables

Answer 51

Proportion of correct decisions (HIT RATE)

Question 52

five items of information must be known in Brogden-Cronbach-Gleser Utility Formula

Answer 52

- Number of employees hired per year - Average tenure - Test validity -Standard deviation of performance in dollars - Mean standardized predictor score of selected applicants

Question 53

- One form of predictive bias - meaning that the test will significantly predict performance for one group and not others.

Answer 53

single-group validity

Question 54

applicants are rank-ordered on the basis of their test scores

Answer 54

top-down selection

Question 55

names of the top three scorers are given to the person making the hiring decision - often used in public sector

Answer 55

rule of three

Question 56

- are a means for reducing adverse impact and increasing flexibility - The minimum test score that an applicant must achieve to be considered for hire.

Answer 56

Passing scores

Question 57

A selection strategy in which applicants must meet or exceed the passing score on more than one selection test

Answer 57

Multiple-cutoff approach

Question 58

Selection practice of administering one test at a time so that applicants must pass that test before being allowed to take the next test.

Answer 58

Multiple-hurdle approach

Question 59

- As a compromise between top down hiring and passing scores, _______ attempts to hire the top test scorers while still allowing some flexibility for affirmative action - A statistical technique based on the standard error of measurement that allows similar test scores to be grouped

Answer 59

banding

Question 60

The number of points that a test score could be off due to test unreliability.

Answer 60

Standard error of measurement (SEM)