Test Development

Question 1

An emerging social phenomenon or pattern of behavior might serve as the stimulus for the development of a new test or in response to a need to assess mastery in emerging occupations or professions.

Answer 1

Test Conceptualization

Question 2

Criterion-referenced testing and assessment are commonly employed in _ and _ contexts.

Answer 2

Licensing
Educational context

Question 3

The items that best discriminate between 2 groups would be considered the _ items.

Answer 3

Good items

Question 4

A good items on a _ test is an item for which high scorers on the test respond correctly and low scorers respond incorrectly.

Answer 4

Norm-referenced test

Question 5

The preliminary research surrounding the creation of a prototype of the test. It should be done to evaluate whether they should be included in the final form of the instrument.

Answer 5

Pilot work

Question 6

The process by which a measuring device is designed and calibrated and by which numbers are assigned to different amounts of trait, attribute or characteristics being measured.

Answer 6

Scaling

Question 7

He is the one credited for being at the forefront efforts to develop methodologically sound scaling methods.

Answer 7

L. L. Thurstone

Question 8

Types of scales

Answer 8

Age-based scale
Grade-based scale
Stanine scale

Question 9

A type of scale where all raw scores on the test are to be transformed into scores that can range from 1-9.

Answer 9

Stanine scale

Question 10

The 3 scaling methods

Answer 10

Rating Scale
Summative scale
Likert scale

Question 11

A grouping of words, statements or symbols on which judgments of the strength of a particular trait, attitude of emotion are indicated by the test taker.

Answer 11

Rating Scale

Question 12

Test score is obtained by summing the rating across all the items.

Answer 12

Summative scale

Question 13

A type of summative rating scale that is used extensively in psychology to scale attitudes. Each items present the testtakers with five alternative responses usually on an agree-disagree or approve-disappaprove continuum.

Answer 13

Likert scale

Question 14

When one dimension is presumed to underlie the ratings.

Answer 14

Unidimensional

Question 15

When more than 1 dimension is thought to guide the testtaker’s responses.

Answer 15

Multidimensional

Question 16

What are the 4 scaling methods that produce ordinal data?

Answer 16

Method of paired comparison
Comparative scaling
Categorical scaling
Guttman scale

Question 17

A scaling method that produces ordinal data. Testtakers are presented with pairs of stimuli which they are asked to compare, and they must select one of the stimuli according to some rule. Then they receive a higher score for selecting the option deemed more justifiable by the majority of a group of judges.

Answer 17

Method of Paired comparison

Question 18

A scaling method that produces ordinal data. Stimuli such as printed cards, drawings, photographs or other objects are typically presented to testtakers for evaluation and must be sort from most justifiable to least justifiable. It could also be accomplished through the use of list of items on a sheet of paper.

Answer 18

Comparative scaling

Question 19

A scaling method that produces ordinal data. Stimuli are placed into one of two or more alternative categories that differ quantitatively with respect to some continuum.

Answer 19

Categorical scaling

Question 20

A scaling method that produces ordinal data. Items on it range sequentially from weaker to stronger expressions of the attitude, belief or feeling being measured. All respondents who agree with the stronger statements of the attitude will also agree with milder statements.

Answer 20

Guttman scale

Question 21

The resulting data of Guttman scale are analyzed by the means of this. This is an item-analysis procedure and approach to test development that involves a graphic mapping of a testtaker’s responses.

Answer 21

Scalogram Analysis

Question 22

The reservoir from which the item will or will not be drawn for the final version of the test. Items available for use as well as new items created especially for the item bank.

Answer 22

Item pool

Question 23

It is the form, plan, structure, arrangement and layout of individual test items.

Answer 23

Item format

Question 24

The two types of item format:

Answer 24

Selected response format
Constructed response format

Question 25

It requires testtakers to select a response from a set of alternative responses.

Answer 25

Selected response format

Question 26

3 Types of selected response format:

Answer 26

Multiple choice format Matching item True-false

Question 27

Several incorrect alternatives or options in a multiple choice format are referred to as _.

Answer 27

Distractors or foils

Question 28

A selected response format where the testtaker is presented with 2 columns where they have to determine which response is best associated with which premise.

Answer 28

Matching item

Question 29

A multiple choice item format that contains only two possible responses (binary choice) (agree or not, yes or no, right or wrong, fact or opinion). It usually takes the form of a sentence.

Answer 29

True-false

Question 30

3 types of constructed response items:

Answer 30

Completion item Short-answer item Essay

Question 31

A constructed response format that requires the examinee to provide a word or phrase that completes a sentence.

Answer 31

Completion item

Question 32

A constructed response format where a word, term, sentence or paragraph may qualify as an answer.

Answer 32

Short-answer item

Question 33

A constructed response format that requires the testtaker to respond to a question by writing a composition, typically one that demonstrates recall of facts, understanding, analysis and/or interpretation.

Answer 33

Essay

Question 34

A relatively large and easily accessible collection of test questions.

Answer 34

Item bank

Question 35

An interactive, computer-administered test-taking process wherein items presented to the testtaker are based in part on the test takers' performance on previous items.

Answer 35

Computerized adaptive testing

Question 36

It refers to the diminished utility of an assessment tool for distinguishing testtakers at the low end of the ability, trait or other attribute being measured. Testtakers who have not yet achieved such ability might fail all the items.

Answer 36

Floor effect

Question 37

It refers to the diminished utility of an assessment tool for distinguishing testtakers at the high end of the attribute being measured. It is likely that the test users who answered all of the items correctly conclude that the test was too easy.

Answer 37

Ceiling effect

Question 38

The ability of the computer to tailor the content and order of presentation of test items on the basis of responses to the previous items. Random presentation of test items.

Answer 38

Item branching

Question 39

What are the 3 different scoring models?

Answer 39

Cumulative model Class scoring or Category scoring Ipsative scoring

Question 40

Scoring model where the higher the score on the test, the higher the testtakers are on the ability or characteristic that the test purpots to measure.

Answer 40

Cumulative model

Question 41

Scoring model where testtakers earn credit toward placement in a particular class or category with other testtaker whose pattern of responses is presumably similar in some way. Used by some diagnostic systems.

Answer 41

Class scoring or Category scoring

Question 42

Scoring model that compares a testtaker's score on one scale within a test to another scale within that same test.

Answer 42

Ipsative scoring

Question 43

The informal rule of thumb for test tryout is that there should be no fewer than _ subjects and preferably as many as _ for each item on the test.

Answer 43

5 10

Question 44

Factors that actually are just artifacts of the small sample size.

Answer 44

Phantom factors

Question 45

A lowercase itallic "p" is used to denote _.

Answer 45

Item Difficulty

Question 46

The larger the item difficulty index, the _ the item.

Answer 46

Easier

Question 47

The optimal average item difficulty for maximum discrimination among the abilities of testtakers.

Answer 47

Approximate 0.5

Question 48

The range of difficulty for individual items on the test.

Answer 48

0.3-0.8

Question 49

For the possible effect of guessing, the optimal average item difficulty is usually the midpoint between _ and the chance success proportion.

Answer 49

1.00

Question 50

The probability of answering correctly by random guessing.

Answer 50

Chance success proportion

Question 51

The higher the index, the greater the test's _.

Answer 51

Internal consistency

Question 52

A statistical tool useful in determining whether items on a test appear to be measuring the same thing.

Answer 52

Factor analysis

Question 53

It is a statistic designed to provide an indication of the degree to which a test is measuring what it purports to measure.

Answer 53

Item-validity index

Question 54

The higher the item validity index, the greater the test's _.

Answer 54

Criterion-related validity

Question 55

It compares performance on a particular item with performance in the upper and lower regions of a distribution of continuous test scores.

Answer 55

Item discrimination index

Question 56

Item discrimination index is symbolized by _.

Answer 56

Lowercase itallic "d"

Question 57

The _ the value of d, the more adequately the item discriminates the higher-scoring from the lower-scoring testtakers.

Answer 57

Higher

Question 58

The highest possible value of d.

Answer 58

+/-1.00

Question 59

The value of d that indicates the item is not discriminating for there is the same proportion of members of the upper and lower groups who pass the item.

Answer 59

0

Question 60

The lowest value that an index of item discrimination can take. It indicates that all members of the upper group failed the item and all members of the lower group passed it.

Answer 60

- -1

Question 61

It is a graphic representation of item difficulty and discrimination. The steeper the slope, the greater the item discrimination.

Answer 61

Item-characteristic curves

Question 62

It is an item that favors one particular group of examinees in relation to another when differences in group ability at e controlled.

Answer 62

Biased test item

Question 63

It is exemplified by different shapes of item-characteristic curves for different groups when the 2 groups do not differ in total test score.

Answer 63

Differential item functioning

Question 64

These are techniques of data generation and analysis that rely primarily on verbal rather than mathematical or statistical procedures.

Answer 64

Qualitative methods

Question 65

It is a general term for various no statistical procedures designed to explore how individual test items work. Involves exploration of the issues through verbal means.

Answer 65

Qualitative item analysis

Question 66

A qualitative research tool designed to shed light on the testtaker's thought processes during the administration of a test. They are asked to think aloud as they respond to each item.

Answer 66

"Think aloud" test administration

Question 67

If a study of test items typically conducted during the test developmental processes in which the items are examined for fairness to all prospective testtakers and for the presence of offensive language, stereotypes or situations.

Answer 67

Sensitivity review

Question 68

It refers to the revalidation of a test on a sample of testtakers other than those of whom test performance was originally found to be a valid predictor of some criterion.

Answer 68

Cross validation

Question 69

The decrease in item validitities that inevitably occurs after cross-validation of finding.

Answer 69

Validaty shrinkage

Question 70

Test validation process conducted on two or more tests using the same sample of testtakers.

Answer 70

Co-validation

Question 71

Used in conjunction with the creation of norms or the revision of existing norms.

Answer 71

Co-norming

Question 72

The discrepancies of the scorers are resolved by another scorer which is called?

Answer 72

Resolver

Question 73

Scored by a highly authoritative scorer that is designed as a model for scoring and a mechanism for resolving scoring discrepancies.

Answer 73

Anchor protocol

Question 74

Discrepancy between scoring in an anchor protocol and the scoring of another protocol.

Answer 74

Scoring drift

Question 75

Those items that respondents from different groups at the same level have underlying trait have different probabilities of endorsing as a function of their group membership.

Answer 75

Differential item functioning (DIF) items