Test Construction

Question 1

Question ID #18254: Assuming no constraints in terms of time, money, or other resources, the best (most thorough) way to demonstrate that a test has adequate reliability is by using which of the following techniques? 
Select one: 
A. equivalent (alternate) forms 
B. test-retest  
C. Cronbach's alpha 
D. Cohen's kappa

Answer 1

A

Question 2

Question ID #18290: When a test user uses a correction for guessing formula that involves subtracting points from each examinee’s scores, the resulting distribution of scores will have a ____________________ than the original (non-corrected) distribution.
Select one:
A. higher mean and larger standard deviation
B. higher mean and smaller standard deviation
C. lower mean and larger standard deviation
D. lower mean and smaller standard deviation

Answer 2

C

Question 3

Question ID #18268: You ask a group of experienced salespeople to review the test items included in a test you have developed to help select new sales applicants. You are apparently interested in determining the test's \_\_\_\_\_\_ validity. 
Select one: 
A. incremental 
B. content  
C. concurrent 
D. differential

Answer 3

B

Question 4

Question ID #18266: When using criterion-referenced interpretation of scores obtained on a job knowledge test, you would most likely be interested in which of the following?
Select one:
A. the total number of test items answered correctly by an examinee
B. an examinee’s performance relative to that of other examinees
C. an examinee’s standing on two or more measures designed to assess the same characteristic
D. ensuring that test items are based on a systematic job evaluation

Answer 4

A

Question 5

Question ID #18238: Which of the following scores does not "belong with" the other three? 
Select one: 
A. stanine scores 
B. z-scores 
C. percentile ranks 
D. percentage scores

Answer 5

D

Question 6

Question ID #18270: The minimum and maximum values of the standard error of estimate are:
Select one:
A. -1 and +1.
B. 0 and 1.
C. 0 and the standard deviation of the predictor.
D. 0 and the standard deviation of the criterion.

Answer 6

D

Question 7

Question ID #18663: A test developer would construct an expectancy table to:
Select one:
A. facilitate norm-referenced interpretation of test scores.
B. facilitate criterion-referenced interpretation of test scores.
C. correct obtained scores for the effects of guessing.
D. correct obtained test scores for the effects of measurement error.

Answer 7

B

Question 8

Question ID #455: It would be most important to assess the test-retest reliability of a measure that:
Select one:
A. is subjectively scored.
B. assesses examinees’ speed of responding.
C. measures a stable trait.
D. measures a characteristic that fluctuates over time.

Answer 8

C

Question 9

Question ID #18228: The correction for attenuation formula is used to measure the impact of increasing:
Select one:
A. a tests reliability on its validity.
B. a tests validity on its reliability.
C. the number of test items on the tests validity.
D. the number of test items on the tests reliability.

Answer 9

A

Question 10

Question ID #18662: In the context of test construction, cross-validation is associated with which of the following? 
Select one: 
A. shrinkage  
B. criterion deficiency 
C. criterion contamination 
D. banding

Answer 10

A

Question 11

Question ID #18283: After reviewing the data collected on a new selection test during the course of a criterion-related validity study, a psychologist decides to lower the selection test cutoff score. Apparently the psychologist is hoping to do which of the following?
Select one:
A. reduce the number of false negatives
B. increase the number of true positives
C. reduce the number of false positives
D. increase the number of false negatives

Answer 11

B

Question 12

Question ID #18235: An advantage of using the kappa statistic rather than percent agreement when assessing a test’s inter-rater reliability is that the former:
Select one:
A. is easier to calculate.
B. corrects for chance agreement.
C. corrects for small sample size.
D. takes into account the effects of multicollinearity.

Answer 12

B

Question 13

Question ID #18661: A reliability coefficient of .60 indicates that \_\_\_ of variability in test scores is true score variability. 
Select one: 
A. 60%  
B. 40% 
C. 36% 
D. 16%

Answer 13

A. A reliability coefficient is interpreted directly as a measure of true score variability. A reliability coefficient of .60 indicates that 60% of the variability in scores is true score variability, while the remaining 40% of the variability is due to measurement (random) error.

Question 14

Question ID #18262: Incremental validity is a measure of:
Select one:
A. decision-making accuracy.
B. shrinkage
C. the generalizability of research results.
D. the costs involved in using a predictor.

Answer 14

A

Question 15

Question ID #18226: To maximize the ability of a test to discriminate among test takers, a test developer will want to include test items that vary in terms of difficulty. If the test developer wants to add more difficult items to her test, she will include items that have an item difficulty index of: 
Select one: 
A. .90. 
B. .50.  
C. .10. 
D. 0

Answer 15

C

Question 16

Question ID #656: In terms of item response theory, the slope (steepness) of the item characteristic curve indicates the item’s:
Select one:
A. level of difficulty.
B. ability to discriminate between examinees.
C. internal consistency reliability.
D. criterion-related validity.

Answer 16

B

Question 17

Question ID #18257: In factor analysis, when two factors are “orthogonal,” this means that:
Select one:
A. the factors are correlated.
B. the factors are uncorrelated.
C. the factors explain a statistically significant amount of variability in test scores.
D. the factors do not explain a statistically significant amount of variability in test scores.

Answer 17

B

Question 18

Question ID #18267: A test developer would use the Kuder-Richardson Formula (KR-20) in order to:
Select one:
A. evaluate a tests internal consistency reliability.
B. evaluate a tests test-retest reliability.
C. determine the impact of increasing a tests reliability on its validity.
D. determine the impact of lengthening a test on its reliability.

Answer 18

A

Question 19

Question ID #18229: When a test has been constructed on the basis of item response theory, an examinee’s total test score provides information about his/her:
Select one:
A. status on a latent trait or ability.
B. predicted performance on an external criterion.
C. performance relative to other examinees included in the standardization sample.
D. current developmental level.

Answer 19

A

Question 20

Question ID #18660: Which of the following is used to estimate the effects of shortening or lengthening a test on the test's reliabilty coefficient? 
Select one: 
A. Cohen's kappa statistic 
B. Kuder-Richardson Formula 20 
C. Cronbach's coefficient alpha 
D. Spearman-Brown formula

Answer 20

D

Question 21

Question ID #18256: In factor analysis, a factor loading indicates the correlation between:
Select one:
A. a test and an identified factor.
B. two different tests.
C. two factors measured by the same test.
D. two factors measured by different tests.

Answer 21

A