Chapter 5

Question 1

Potentially start earlier in chapter

Answer 1

check

Question 2

Classical test theory/true score theory

Answer 2

A system of assumptions about measurement that includes the notion that a test score and a response to an individual item is composed of a relatively stable component that actually is what the test or individual item is designed to measure, as well as a component that is error

Question 3

True score

Answer 3

According to CTT, a value that genuinely reflects an individual’s ability level as measured by a particular test- is test dependent

Question 4

Advantages of CTT

Answer 4

Assumptions easy to meet, simple, compatible and easy to use with widely used statistical techniques

Question 5

Problems with CTT

Answer 5

All items presumed to be contributing equally to score- questionable assumption, assumptions favor longer tests

Question 6

Domain sampling theory

Answer 6

A system of assumptions about measurement that includes the notion that a test score consists of a relatively stable component that actually is what the test (or item) is designed to measure as well as relatively unstable components that collectively can be accounted for as error
??

Question 7

Generalizability theory

Answer 7

Based on idea that person’s test scores vary from testing to testing because of variables in the testing situation

Question 8

Universe

Answer 8

Total context of a particular test situation, including all factors that lead to an individual test taker’s score

Question 9

Facets

Answer 9

Variables of interest in the universe, ie number of items in the test, amount of training test scorers have, purpose of test administration

Question 10

Universe score

Answer 10

Test score that should be obtained given the exact same conditions of all the facets in the universe

Question 11

Generalizability study

Answer 11

Examines how generalizable scores from a particular test are if the test is administered in different situations, examines how much of an impact different facets of the universe have on the test score

Question 12

Coefficients of generalizability

Answer 12

Influence of particular facets on the test score

Question 13

Decision study

Answer 13

Examine usefulness of test scores in helping the test user make decisions

Question 14

Item response theory / latent-trait theory

Answer 14

A family of theories and methods that provide a way to model the probability that a person with X ability will be able to perform at a level of Y (with X amount of trait, will exhibit Y amount of trait on test designed to measure it); a system of assumptions about measurement including that a trait being measured is unidimensional and extent to which each test item measures the trait??

Question 15

Discrimination

Answer 15

In context of IRT, the degree to which an item differentiates among people with higher or lower levels of what is being measured

Question 16

Dichotomous test items

Answer 16

Test items that can be answered with only one of two alternative responses

Question 17

Polytomous test items

Answer 17

Test items with three or more alternative responses, where only one is scored correct or consistent with a targeted construct

Question 18

Assumptions in latent trait models

Answer 18

Something about underlying frequency distribution of test scores?

Question 19

Rasch model

Answer 19

IRT model with specific assumptions about underlying distribution

Question 20

Standard error of measurement

Answer 20

Measure of precision of an observed test score, provides an estimate of the amount of error inherent in an observed score or measurement, inverse relationship with reliability of test; tool used to estimate the extent to which an observed score deviates from a true score, aka standard error of a score, index of the extent to which one individual’s scores vary over tests presumed to be parallel
do I need to know this formula

Question 21

Confidence interval

Answer 21

A range or band of test scores that is likely to contain the true score

Question 22

Standard error of the difference

Answer 22

A statistical measure that can aid a test user in determining how large a difference should be before it is considered statistically significant
formula?