Chapter 5 Flashcards
(90 cards)
1
Q
refers to something that produces similar results
A
reliability
2
Q
a statistic that quantifies reliability which ranges from 0 to 1
A
reliability coefficient
3
Q
what is the score for not reliable at all
A
zero (0)
4
Q
what is the score for perfectly reliable
A
one (1)
5
Q
the individual’s score on a measure if there was no error
A
true score
6
Q
a person’s standing on the theoretical variable independent of any particular measurement
A
construct score
7
Q
______ : reliability :: _____ : validity
A
true score; construct score
8
Q
formula for observed score
A
X = T + E
9
Q
refers to the difference between the observed score and the true score
A
error
10
Q
standard deviation squared
A
variance
11
Q
variance equals _____ plus _____
A
true variance; error variance
12
Q
the proportion of the total variance attributed to true variance
A
reliability
13
Q
percentage of true variance
A
67%
14
Q
percentage of error due to test construction
A
18%
15
Q
percentage of administration error
A
5%
16
Q
percentage of unidentified error
A
5%
17
Q
percentage of scorer error
A
5%
18
Q
refers the inherent uncertainty associated with any measurement, even after care has been taken to minimize preventable mistakes
A
measurement error
19
Q
consists of unpredictable fluctuations and inconsistencies of other variables in the measurement process
A
random error
20
Q
typically proportionate to what is presumed to be the true value of the variable being measured
A
systematic error
21
Q
sources of error variance
A
- test construction
- test administration
- test scoring and interpretation
22
Q
what are the variables under test administration source of error variance
A
- test taker variables
- examiner-related variables
23
Q
sources of error variance: variation may exist within items in a test or between tests
A
test construction
24
Q
sources of error variance: may stem from the test environment
A
test administration
25
sources of error variance: pressing emotional problems, physical discomfort, lack of sleep, and the effects of drugs or medication
test taker variables
26
sources of error variance: physical appearance and demeanor may play a role
examiner-related variables
27
sources of error variance: computer testing reduces error in test scoring, but many tests still require expert interpretation
test scoring and interpretation
28
reliability estimates
- test-retest reliability
- split-half reliability
- inter-scorer reliability
29
an estimated of reliability obtained by correlating pairs of scores from the same people on two different administrations of the same test
test-retest reliability
30
true or false: in test-retest reliability, as time passes, correlation between the scores obtained on each testing increases
false; decreases
31
this reliability is most appropriate for variables that should be stable over time, such as personality
test-retest reliability
32
with intervals greater than 6 months, the estimate of test-retest reliability is called _____
coefficient of stability
33
measures the degree of the relationship between various forms of a test by means of alternate-forms or parallel-forms
coefficient of equivalence
34
for each form of the test, the means and the variances of observed test scores are equal
parallel forms
35
typically designed to be equivalent with respect to variables such as content and level of difficulty
alternate forms
36
obtaining estimates of alternate-forms reliability and parallel-forms reliability is similar to obtaining an estimate of _____
test-retest reliability
37
obtained by correlating two pairs of scores obtained from equivalent halves of a single test administered once
split-half reliability
38
step 1 of split-half reliability
divide the test into equivalent halves
39
step 2 of split-half reliability
calculate a pearson r between scores on the two halves of the test
40
step 3 of split-half reliability
adjust the half-test reliability using the spearman-brown formula
41
allows a test developer or user to estimate internal consistency reliability from a correlation of two halves of a test
spearman-brown formula
42
the degree of correlation among all the items on a scale
inter-item consistency
43
mean of all possible split-half correlations
coefficient alpha
44
range of coefficient alpha
0 to 1
45
coefficient alpha is corrected by what formula
spearman-brown formula
46
the degree of agreement or consistency between two or more scorers with regard to a particular measure
inter-scorer reliability
47
what reliability is often used when coding nonverbal behavior
inter-scorer reliability
48
a correlation coefficient used to determine the degree of consistency among scorers
coefficient of inter-scorer reliability
49
the nature of tests
- homogeneity vs. heterogeneity of items
- dynamic vs. static characteristics
- restriction of range vs. inflation of range
- speed test vs. power test
50
estimates the portion of a test score that is attributable to error
true score theory or classical test theory
51
averaging all the observed scores obtained over a period of time, the result would be closest to the true score
true score theory or classical test theory
52
the greater the number of items, the higher the reliability
true score theory or classical test theory
53
estimates the extent to which specific sources of variation under defined conditions are contributing to the test score
domain sampling theory
54
assumes that the items that have been selected for any one test are just a sample of items from an infinite domain of potential items
domain sampling theory
55
based on the idea that a person's test scores vary from testing to testing because of variables in the testing situation
generalizability theory
56
it is described in terms of its facets
universe
57
provides a way to model the probability that a person with x ability will be able to perform at a level of y
item response theory
58
refers to a family of methods and techniques used to distinguish specific approaches
item response theory
59
incorporates considerations of an item's level of difficulty and discrimination
item response theory
60
relates to an item not being easily accomplished, solved, or comprehended
difficulty
61
refers to the degree to which an item differentiates among people with higher or lower levels of the variables being measured
discrimination
62
the _____ the reliability of the test, the _____ the standard error
higher; lower
63
the higher the _____ of the test, the lower the _____
reliability; standard error
64
can be used to estimate the extent to which an observed score deviates from a true score
standard error
65
a range or band of test scores that is likely to contain the true score
confidence interval
66
percent between +-1 sd
68.3%
67
percent between +-2 sd
95.4%
68
percent between +-3 sd
99.7%
69
percent 1 sd from the mean
34.1%
70
percent 2 sd from the mean
13.6%
71
percent 3 sd from the mean
2.1%
72
when a time limit is long enough to allow test takers to attempt all items, and if some items are so difficult that no test taker is able to obtain a perfect score
power test
73
generally contains items of uniform level of difficulty so that, when given generous time limits, all test takers should be able to complete all the test items correctly
speed test
74
designed to provide an indication of where a test taker stands with respect to some variable or criterion
criterion-referenced test
75
examines how generalizable scores from a particular test are if the test is administered in different situations
generalizability study
75
error that is unpredictable
random error
76
error that is expected so you have prepared for it
systematic error
77
summary of test-retest reliability
test-retest = different administrations - coefficient of stability
78
summary of parallel/alternate forms
parallel/alternate forms = different forms - coefficient of equivalence
79
summary of split-half reliability
split-half reliability = different halves of the test - person r (correlation) & spearman brown (adjustment)
80
score for reliability
0.8 and above
81
all items measure only one construct
homogenous items
82
all items measures lots of constructs
heterogenous items
83
in which items is internal consistency high
homogenous items
84
what coefficient removes biases in scoring
coefficient of inter-scorer reliability
85
analyze correlation at a specific range; high reliability
restriction of range
86
looks at the whole picture; low reliability
inflation of range
87
what is measured in power test
ceiling and floor limits
88
highest that you can analyze
ceiling limits
89
lowest that you can understand
floor limits
