PSY Exam 2

Question 1

What is Reliability?

Answer 1

the degree which test scores are free from errors of measurement (consistency)

Question 2

What is Observed Test Score?

Answer 2

true score plus error (X = T + E)

Question 3

What is the Classical Test Theory?

Answer 3

No instrument is perfectly reliable or consistent and all test scores contain some error

Question 4

What is Measurement Error?

Answer 4

variations in measurement using a reliable instrument

Question 5

What is a Reliable Test?

Answer 5

A test we can trust to measure each person in approximately the samew ay every time it is used

Question 6

What are the two types of error?

Answer 6

Random and Systematic

Question 7

What is Random Error?

Answer 7

error that is random in nature that lowers the reliability of a test (will lower a test score by exactly the same amount with an infinite amount of testing)

Question 8

What is Systematic Error?

Answer 8

error that occurs when a source of error always changes a true score but does not lower the reliability of a test (since the test is already reliably inaccurate it affects descriptive statistics)

Question 9

What is the Reliability Coefficient?

Answer 9

an index of the strength of the relationship between two sets of scores (think of spearman-brown formula)

Question 10

What are the 3 categories of Reliability Coefficients?

Answer 10

Test-Retest, Alternate Forms, Internal Consistency

Question 11

What is Test-Retest reliability?

Answer 11

A test is taken at two different points in time and are compared while facing the challenge of practice effects and fatigue (looking for correlation between scores)

Question 12

What is Alternate Forms reliability?

Answer 12

A test developer creates two (or more) different forms of the test (order effects & parallel forms)

Question 13

What is Internal Consistency reliability?

Answer 13

A measure of how related items or groups of items on a test are to each other (appropriate for homogeneous tests)

Question 14

What is a Homogeneous Test?

Answer 14

a test that is measuring only one trait or characteristic

Question 15

What is a Heterogeneous Test?

Answer 15

a test that is measuring more than one trait or characteristic

Question 16

What is Scorer Reliability and Agreement?

Answer 16

the amount of consistency among scorers’ judgments (two or more individuals score the same test)

Question 17

What is interscorer agreement?

Answer 17

the amount of consistency among scorers’ judgments

Question 18

What is intrascorer reliability?

Answer 18

whether a person is consistent in the way they assigned scores from test to test

Question 19

What is Interrater agreement?

Answer 19

an index of how consistently the scorers rate or make decisions

Question 20

What is Intrarater agreement?

Answer 20

when one scorer makes consistent judgments across all tests

Question 21

What is KR-20 used for?

Answer 21

Estimating the internal consistency for tests whose questions can scored either right or wrong? (ex: true or false)

Question 22

What is coefficient alpha used for?

Answer 22

Estimating the internal consistency tests whose questions have more than two possible answers (ex: rating scales)

Question 23

What is Cohen’s Kappa?

Answer 23

an index for calculating scorer reliability / inter-rater agreement when scorers make judgments that result in nominal and ordinal data (-1.0 to 1.0)

Question 24

What is standard error of measurement?

Answer 24

an index of the amount of uncertainty or error expected in an individual’s observed test score… how much the individual’s observed score might differ from the individual’s true test score.

Question 25

What is the standard error of measurement formula?

Answer 25

SEM = SD * √(1 - r) | standard error of measurement = standard deviation multiplied by the square root of 1 minus the reliability coefficient of the test

Question 26

What is a confidence interval?

Answer 26

the range of scores that we feel confident will include the test taker's true score ±1 STD dev of the mean → 68.7 ±2 STD dev of the mean → 95% ±3 STD dev of the mean → 99%

Question 27

What are the four factors that influence reliability?

Answer 27

test itself, test administration, test scoring, and test takers

Question 28

What are the 6 factors related to the four sources of test error?

Answer 28

Test length, homogeneity of test questions, test-retest interval, test administration, scoring, and cooperation of test takers

Question 29

What is the Generalizability Theory?

Answer 29

an approach for estimating reliability that is concerned with how well and under what conditions can we generalize an estimation of reliability of test scores from one test administrator to another

Question 30

What did Dale say about G Theory?

Answer 30

it is a way of saying we know for reliability you have error (systematic / random) but now there are all of these sources and it will be nice to break apart these sources of error

Question 31

What is validity?

Answer 31

the extent to which a test accurately measures what it is intended to measure