Test Development

Question 1

It is the product of the thoughtful and sound application of established principles of test construction

Answer 1

Test development

Question 2

1st step of Test development

Answer 2

Test conceptualization (what, how, who, when, should?)

Question 3

Preliminary research surrounding the creation of a prototype of the test

Answer 3

Pilot study/research

Question 4

2nd step of test development

Answer 4

Test construction

Question 5

Process of setting rules for assigning number in measurement

Answer 5

scaling

Question 6

Credited for being at the forefront of efforts to develop methodologically sound scaling methods

Answer 6

LL Thurstone

Question 7

Type of scale the consists grouping of words, statement, symbols on which judgments of the strength of a particular trait, attitude, emotion are indicated by the test-taker

Answer 7

Rating scale

Question 8

A scale where the final score is obtained by summing the ratings across all items (e.g. Likert Scale)

Answer 8

Summative scale

Question 9

A scale where test takers are presented with pairs of stimuli which they are asked to compare

Answer 9

Method of paired comparison

Question 10

Entails sorting tasks and judgments of a stimulus in comparison with every other stimulus on the scale (e.g. sort items from most justifiable to least justifiable)

Answer 10

Comparative scaling (ordinal)

Question 11

Stimuli placed into one of two or more alternative categories that differ quantitatively with respect to some continuum

Answer 11

categorical scaling

Question 12

Respondents who agree with stronger statements of the attitude will also agree with the milder statements

Answer 12

Guttman scale (ordinal)

Question 13

Item analysis procedure and approach to test development that involves a graphic mapping of a testtaker’s responses

Answer 13

Scalogram analysis

Question 14

Scaling method used to obtain data that are presumed to be in interval in nature

Answer 14

Equal-appearing intervals (thurstone)

Question 15

Reservoir or from which items will or will not be bdrawn for the final version of test

Answer 15

Item pool

Question 16

Parts of a multiple-choice item format question

Answer 16

stem (sentence)
correct option
distractors/foils

Question 17

Also called as short-answer item

Answer 17

Completion item

Question 18

Limitations of essay items

Answer 18

Focus on a liimited area; subjectivity in scoring

Question 19

Relatively large and easily accessible collection of test questions

Answer 19

item bank

Question 20

Interactive, computer-administered test taking process wherein items presented to the testtaker are based in part on the testtaker’s performance on previous items

Answer 20

Computerized-adaptive testing (CAT)

Question 21

Ability of the computer to tailor the content and order of the presentation of test items on the basis of responses to previous items

Answer 21

Item branching

Question 22

Most commonly used scoring model

Answer 22

Cumulative scoring

Question 23

A type of scoring used by some diagnostic systems wherein individuals must exhibit a certain number of symptoms to qualify to a specific diagnosis

Answer 23

Class/categorical scoring

Question 24

Compare testtaker’s score on one scale within a test to another scale within that same test

Answer 24

Ipsative scoring

Question 25

3rd step in test development

Answer 25

test tryout

Question 26

4th step in test development

Answer 26

Item analysis

Question 27

Items that spur motivation and positive testtaking attitude and lessen anxiety

Answer 27

Give away items

Question 28

Percent of people who said yes, agreed, endorsed the item not who pass the item

Answer 28

Item endorsement index

Question 29

Range of the optimal item difficulty

Answer 29

0.3-0.8(easy)

Question 30

Formula for OID

Answer 30

chance performance +1/2

Question 31

OID for true-false item

Answer 31

0.75 (chance=0.5)

Question 32

OID for multiple choice item 4 options

Answer 32

0.63 (chance=0.25)

Question 33

OID for multiple choice item 5 options

Answer 33

0.60 (chance=0.2)

Question 34

Equal to the product of the item-score standard deviation and the correlation between the item score and the total test score

Answer 34

Item reliability index

Question 35

Item Analysis Technique for Questions with right/wrong answers

Answer 35

Item Difficulty Item Discrimination Distractor Analysis

Question 36

Item Analysis Techniques for either right/wrong answers or self-report scales

Answer 36

Item reliability index Cronbach's alpha

Question 37

Equal to the item score SD and correlation between item score and criterion score

Answer 37

Item validity index

Question 38

How adequately an item separates or discriminates between high scorers and low scorers on the entire test

Answer 38

Item discrimination index

Question 39

What are the key properties of the Item-discrimination index?

Answer 39

Symbolised by d * Compares performance on a particular item by the high ability group & the low ability group (i. e. the top 27% and the bottom 27%) * Items that discriminate well will have a high positive score (to a maximum of 1) * A negative d value is a red flag as it means low test takers are doing better on that item than high test takers

Question 40

The quality of each alternative within a multiple choice item can be readily assessed with reference to the comparatives performance of upper and lower scorers

Answer 40

Analysis of item alternatives (test developer can get an idea of the effectiveness of a distractor by means of a simple EYEBALL Test

Question 41

Graphic representation of item difficulty and item discrimination

Answer 41

Item characteristic curve (the steeper the slope, the greater the item discrimination)

Question 42

Test developer addresses the problem of guessing by including in the test manual...

Answer 42

- explicit instructions regarding this point for the examiner to convey to the examinees (ex. instruct answer only if certain) - specific instructions for scoring and interpretting omitted items

Question 43

Can be used to identify biased items

Answer 43

item characteristic curves

Question 44

Different shapes of item-characteristic curves for different groups when 2 groups do not differ in total test score

Answer 44

Differential item functioning

Question 45

Rely primarily on verbal rather than mathematical procedures to explore how individual test items work

Answer 45

Qualitative item analysis (thru group discussion, interviews)

Question 46

Approach to cognitive assessment entails having respondents verbalize thoughts as they occur

Answer 46

think aloud test administration (one-on-one basis)

Question 47

Conducted during the test development process in which items are examined for fairness to all prospective testtakers and for the presence of offensive language, stereotypes or situations

Answer 47

Sensitivity review

Question 48

last step in test development

Answer 48

test revision

Question 49

Test revision in the life cycle of an existing test

Answer 49

*APA suggests that an existing test be kept in its present form as long as it remains useful but that it should be revised when significant changes in the doman represented or new conditions of test use and interpretation make the test inappropriate for its intended use

Question 50

Revalidation of a test on a sample of testtakers other than those on whom test performance was originally found to be a valid predictor of some criterion

Answer 50

cross validation (key step in test development)

Question 51

Decrease in item validities that inevitable occurs after corss-validation of findings

Answer 51

Validity shrinkage (is expected and integral to test development process)

Question 52

Test validation conducted on 2 or more test using the same sample of testtakers

Answer 52

co-validation (also referred as co-norming)

Question 53

Examiners undergo training of test administration using test manual

Answer 53

Quality assurance

Question 54

A test protocol scored by a highly authoritative scorer that is designed as a model for scoring and a mechanism for resolving scoring discrepancies; ensure consistency in scoring

Answer 54

anchor protocol

Question 55

A discrepancy between scoring in an anchor protocol and the scoring of another protocol

Answer 55

scoring drift

Question 56

Evaluate how well an individual item is working to measure different levels of the underlying construct

Answer 56

IRT information curves

Question 57

Item functions differently in one group of testtakers as compared to another group as compared to another group of testtakers known to have the same level of difficulty of the underlying trait (by culture, gender, age)

Answer 57

Differential item functioning (DIF)

Question 58

Test developers scrutinize group-by-group item response curves looking for DIF items

Answer 58

DIF analysis

Question 59

Items that respondents from different groups at the same level of underlying trait have different probabilities of endorsing a function of their group membership

Answer 59

DIF items

Question 60

An advantage of the response format of the test

Answer 60

Great breadth (cover many topics)