reliability Flashcards
(31 cards)
What is the definition of reliability in psychometric testing?
The stability or consistency of the measurement.
What are the key components that define a ‘good test’?
Clear instructions for administration, scoring, and interpretation, economy in time and money, and measuring what it purports to measure.
What is the role of measurement error in reliability?
Each individual measurement has an element of error such as observer’s error, environmental changes, and participant’s changes.
What is the reliability coefficient?
A statistical measure that indicates the extent to which measurements are consistent or repeatable.
What does a higher coefficient alpha indicate?
The higher the coefficient alpha, the higher the reliability.
What are the two factors that reliability analyses assume test scores reflect?
- Stable characteristics of the individual (true characteristics)
- Chance features of the individual (random measurement error)
What is a true score according to classical test theory?
A value that genuinely reflects an individual’s ability or trait level as measured by a particular test.
What does the formula X = T + e represent in classical test theory?
X = observed score, T = true score, e = measurement error.
In a reliable test, what should the value of e (measurement error) be close to?
Close to 0.
What does a reliable test primarily measure according to classical test theory?
Most of the test scores should result from the measurement of true characteristics (T).
Fill in the blank: A reliable test is defined as one on which test takers will fall in the same positions relative to each other, indicating _______.
[consistency]
True or False: Reliability is only concerned with the true score of an individual.
False.
What is the impact of the number of items on test reliability?
Test scores gain reliability as the number of items increases.
What is measurement error?
Refers to the inherent uncertainty associated with any measurement, even after care has been taken to minimize preventable mistakes.
What causes fluctuations in measurement?
Estimates of a quantity differ each time a measurement is taken, even when procedures are followed perfectly and no obvious mistakes are made.
What is error variance?
Variance from irrelevant, random sources is error variance.
What is true variance?
Variance from true differences is true variance.
What does reliability refer to?
The proportion of the total variance attributed to true variance.
What are the two types of measurement error?
Random error and systematic error.
What is random error?
A source of error in measuring a targeted variable, caused by unpredictable fluctuations and inconsistencies of other variables in the measurement process.
What is systematic error?
A source of error in measuring a variable that is typically constant and proportionate to what is presumed to be the true value of the variable being measured.
