Item Analysis

Question 1

Question ID #12385: Content appropriateness, taxonomic level, and extraneous abilities are factors that are considered when evaluating:
Select one:

A.
a test’s factorial validity.

B.
a test’s incremental validity.

C.
the relevance of test items.

D.
the adequacy of the “actual criterion.”

Answer 1

The correct answer is C.

In the context of test construction, relevance refers to the extent to which test items contribute to achieving the goal of testing. Content appropriateness, taxonomic level, and extraneous abilities are three factors that may be considered when determining the relevance of test items.

Answer A: Factorial validity refers to the extent to which a test has high correlations with factors it is expected to correlate with and low correlations with factors it is not expected to correlate with.

Answer B: Incremental validity refers to the degree to which a test improves decision-making accuracy.

Answer D: The actual criterion refers to the actual (versus ultimate) measure of performance.

Question 2

Question ID #12386: The optimal item difficulty index (p) for items included in a true/false test is:
Select one:

A.
.25.

B.
.50.

C.
.75.

D.
1.0.

Answer 2

The correct answer is C.

One factor that affects the optimal difficulty level of an item is the likelihood that an examinee can choose the correct answer by guessing, with the preferred level being halfway between 100% and the level of success expected by chance alone. For true or false items, the probability of obtaining a correct answer by chance alone is .50. Therefore, the optimal difficulty level for true or false items is .75, which is halfway between 1.0 and .50.

Answers A, B, & D are incorrect.

Question 3

Question ID #12387: For an achievement test item that has an item discrimination index (D) of +1.0, you would expect:
Select one:

A.
high achievers to be more likely than low achievers to answer the item correctly.

B.
low achievers to be more likely than high achievers to answer the item correctly.

C.
moderate achievers to be more likely than high and low achievers to answer the item correctly.

D.
low and high achievers to be equally likely to answer the item correctly.

Answer 3

The correct answer is A.

When all examinees in the upper-scoring group and none in the lower-scoring group answer the item correction, the item discrimination index (D) is equal to +1.0. The item discrimination index (D) is calculated by subtracting the percent of examinees in the lower-scoring group who answer the item correctly from the percent of examinees in the upper-scoring group who answer the item correctly. D ranges in value from -1.0 to +1.0.

Answers B, C, and D: These responses present incorrect findings - high achievers are more likely than low achievers to answer the item correctly.

Question 4

Question ID #12389: The slope (steepness) of an item characteristic curve indicates the item’s:
Select one:

A.
difficulty level

B.
discrimination

C.
reliability

D.
validity

Answer 4

The correct answer is B.

The various item response theory models provide item characteristic curves that provide information on one, two, or three parameters - i.e., difficulty level, discrimination, and probability of guessing correctly. An item’s ability to discriminate between high and low achievers is indicated by the slope of the item characteristic curve - the steeper the slope, the greater the discrimination.

Answers A, C, and D: These responses present incorrect information- the slope of an item characteristic curve indicates the item’s discrimination.

Question 5

Question ID #12390: The item difficulty index (p) ranges in value from:
Select one:

A.
-1.0 to +1.0

B.
-.50 to +.50

C.
0 to +1.0

D.
0 to .50

Answer 5

The correct answer is C.

The item difficulty index (p) indicates the proportion of examinees in the tryout sample who answered the item correctly. The item difficulty index ranges in value from 0 to +1.0, with 0 indicating that none of the examinees answered the item correctly and +1.0 indicated that all examinees answered the item correctly.

Answers A, B, and D: These responses present incorrect information - the item difficulty index ranges in value from 0 to +1.0.