Reliabiity, Validity

Question 1

assumes that each person has a true score that would be obtained if there were no errors in measurement

Answer 1

classical test score theory

Question 2

assumes that the items that have been selected for any one test are just a sample of items from an infinite domain of potential itesms

Answer 2

domain sampling theory

Question 3

process of choosing test items that are appropriate to the content domain of the test

Answer 3

domain sampling

Question 4

another central concept in classical test theory wherein it considers the problems created by using a limited number of items to represent a larger and more complicated construct

Answer 4

domain sampling model

Question 5

using ____, the computer is used to focus on the range of item difficulty that helps assess an individual’s ability level

Answer 5

item response theory

Question 6

degree to which scores from a test are stable and results are consistent

Answer 6

reliability

Question 7

ratio of the variance of the true scores on a test to the variance of the observed scores

Answer 7

reliability coefficient

Question 8

test reliability is usually estimated in one of three ways:

Answer 8

• test-retest method
• parallel forms method
• internal consistency method

Question 9

consistency of the test results are considered when the test is administered on different occasions

Answer 9

test-retest method

Question 10

test across different forms of the test are evaluated

Answer 10

parallel forms method

Question 11

performance of people on similar subsets of items selected from the same form of measure is examined

Answer 11

internal consistency

Question 12

occurs when the first testing session influences scores from the second session

Answer 12

carryover effect

Question 13

compares two equivalent forms of a test that measure the same attribute

Answer 13

parallel forms / equivalent forms reliability

Question 14

determined by dividing the total set of items relating to a construct of interest into halves and comparing the results obtained from the two subsets of items thus created

Answer 14

split-half reliability

Question 15

measure of internal consistency; considered to be a measure of scale reliability

Answer 15

coefficient alpha or cronbach’s alpha

Question 16

used to estimate the reliability of binary measurements

Answer 16

kuder and rischardshon formula 20

Question 17

takes into account chance agreement

Answer 17

kappa statistics

Question 18

allows you to estimated what the correlation between the two halves would have been if each half had been the length of the whole test

Answer 18

spearman-brown formula

Question 19

best method for assessing the level of agreement among several observers

Answer 19

kappa statistic

Question 20

agreement between a test score or measure and the quality it is believed to measure

Answer 20

validity

Question 21

3 types of evidence:

Answer 21

• construct-related
• criterion-related
• content-related

Question 22

mere appearance that a measure has validity

Answer 22

face validity

Question 23

logical rather than statistical

Answer 23

content validity

Question 24

describes the failure to capture important components of a construct

Answer 24

construct underepresentation

Question 25

occurs when scores are influenced by factors irrelevant to the construct

Answer 25

construct-irrelevant variance

Question 26

tells us just how well a test corresponds with a particular criterion

Answer 26

criterion validity evidence

Question 27

standard against which the test is compared

Answer 27

criterion

Question 28

SAT is the predictor and GPA is the criterion

Answer 28

predictive validity evidence

Question 29

correlation expressing the relationship between a test and a criterion

Answer 29

validity coefficient

Question 30

established through a series of activities in which a researcher simultaneously defines some construct and develops the instrumentation to measure it

Answer 30

construct validity evidence

Question 31

obtained when a measure correlates well with other tests to measure the same construct

Answer 31

convergent evidence for validity

Question 32

standardized tests that are designed toc compare and rank test takers in relation to one another

Answer 32

norm-referenced test

Question 33

process of evaluating the learning of students against a set of pre-specified qualities or criteria, without reference to the achievement of otehrs

Answer 33

criterion referenced test