Knowledge Questions Part 2 - Classical Test Theory Flashcards
What types of items and scoring in a questionnaire or test form the focus of classical test
theory?
Categorical items
What is the formal definition of reliability in classical psychometrics?
Reliability here is the ratio of the true score variance and the total variance.
Why is the reliability of a measurement tool often indicated by the letter ρ or r, which is the
symbol for correlation (in the population and the random sample respectively)?
r is the symbol for a correlation.
ρ is the symbol for reliability and functions as the symbol for correlation on the population level.
What does the restriction of range effect entail?
The reliability of a measurement instrument drops as the population becomes more homogeneous.
What does thee attenuation effect entail? How can we correct for this effect?
It entails that the true correlation between two scales is often larger than observed. The true correlation is dependent on the internal reliabilities of the scales.
Why would a progress test consist of so many items?
Because this is the main way to increase reliability without having other statistical drawbacks.
What is the Spearman-Brown formula used for? What input is needed and what output will
this produce?
It is used to infer the test reliability from the reliability of the single items.
Name the four methods for testing the reliability of a test or questionnaire.
Split-Half Reliability
Parallel Test Reliability
Test-Retest Reliability
Internal Consistency
Which of these four methods is very different from the other three? (Tip: error of
measurement.)
The test-retest reliability is special as it uses the exact same test at different time points.
Which two types of reliability testing are applications of the Spearman-Brown
formula? What input does each type require?
Internal Consistency: Requires average item-item correlation and K as input.
Split-Half Reliability: Requires reliability of a test half as input.
What does the ‘standard error of measurement’ (SEM) entail?
True score variance is made up of true variance and some residual/error variance. The square-root of this is the SEM. It is a measure of how accurate our test predicts an individual’s true score.
How can you apply the SEM in order to establish whether there is a difference in the true
scores of two individuals? Which assumptions do we make to do this?
To see whether there is a difference between two individuals, one could calculate the difference between their observed scores and then construct a confidence interval using 2 times the SEM, since there are two people involved that each have their own measurement error. If 0 is included in this interval, we cannot say that there is enough evidence that they are truly different. For these computations to have validity, we need to assume normality of the distribution of the SEM and that the measurement errors of the individuals is uncorrelated.
How can you apply the SEM in order to establish whether there is a difference between two measurements of the same individual?
Here we do the same as when inferring the true difference between two people, except that we should use the test-retest method of calculating the SEM instead of any of the other three.
How does item analysis estimate the reliability of each item in a test or questionnaire?
By indicating the item-rest correlation.
Which results from an item analysis could provide grounds for possibly removing items from
a measuring tool?
The value of Cronbach’s Alpha without this item.
