Prelim 2 prep Flashcards
(36 cards)
What are the differences between True Score Theory, Generalizability Theory, and Item Response Theory?
What is the standard error of measurement?
What are confidence intervals and what do they tell us?
When confidence intervals increase in terms of percentage (i.e., 90% vs 95%), what does that do
to the range of scores it comprises?
What does it mean if a test is valid?
What are the three main categories of validity?
What is: content validity, criterion-related validity, construct validity, ecological validity, external
validity, face validity?
What is the content validity ratio and how is it used to determine content validity of test items?
If more than half the panelists of experts say an item is essential, has content validity, CVR 0 is half, negative if fewer than half
Name the three characteristics of a criterion
Relevant, valid, uncontaminated
What does it mean for a criterion to be uncontaminated?
Independent- independent group of raters decides who is good and who isn’t, then correlate that with test scores
Define concurrent validity and predictive validity
Concurrent- degree to which a test score is related to some criterion measure obtained at the same time
Predictive- degree to which a test score predicts a criterion measure
What are false negatives, false positives, specificity, and sensitivity?
False negative- test predicts someone doesn’t possess a trait and they do
False positive- test says someone has a trait and they don’t
Specificity- perfect wouldn’t mistakenly identify as someone having a trait when they don’t
Sensitivity- perfect identify all people who have the trait
What is incremental validity and what would be proof of its existence?
Extent to which adding a second or third predictor gives more information about a criterion
proof??
What is construct validity?
Extent to which a test measures a construct we are examining
Name a describe the several ways in which you can find evidence for construct validity.
- Homogenous
- Evidence changes w age
- Test scores change w experience
- Distinct groups score differently
- Convergent evidence between two tests measuring the same construct
What is the difference between convergent and concurrent validity?
What is a factor analysis and how does an exploratory factor analysis differ from a confirmatory
one?
?
Exploratory- estimating or extracting factors, deciding how many to retain, rotating to an interpretable orientation ????
Confirmatory- degree to which a hypothetical model fits the data
Name and define the different types of rating error that can occur
Leniency error- arises from tendency on part of rater to be lenient
Severity- opposite
Central tendency- rater doesn’t use extreme ends of scale
Halo effect- seeing people well no matter what
What is test utility?
Usefulness or practical value of testing to improve efficiency, use in a particular situation helps us make better decisions
What are some of the costs of administering a test, and what are some costs of NOT
administering one?
Administering:
- buying
- supply of blank test protocols
- computer program to score the test
- paying to score the test
- hiring people to administer the test
- costs of doing business
Not administering:
- loss of confidence as an ultimate cost of the company???
- missing a child abuser
- failing to diagnose when someone underreports on an interview???
Keep in mind the real-life example I discussed about how to think about the cost of testing when
doing evaluations.
????
What are the Taylor Russel tables used for and what three variables are considered when using
them to decide if giving a test is “worth it.”
- COME BACK
Be able to name a few other tables (i.e., Naylor-Shine) and have a basic sense of how they work
(they could be multiple-choice option for instance)
COME BACK
Name some different ways cut scores are determined.
COME BACK
