Comps - Assessment (Validity) Flashcards
Validity
The degree to which evidence and theory support the interpretations of test scores entailed by the proposed uses of a test
If validity is not a property of a test, what is it?
It is a property of how the clinician uses the test
Tests are not valid in and of itself; __________ made about test scores and use of test scores are more or less valid
Inferences
Content Validity
Examining the match between the content of the test and the content that should be included on a test of that particular attribute (extent to which questions on a test represent the construct being measured)
What is a good question to conceptualize content validity?
How well does the content match what should be on the test?
What do you want to avoid if you want a high content-valid test?
Construct-irrelevant content (items that are not related to the content being measured)
Threats to Content Validity
-Inaccurate knowledge
-Construct underrepresentation (something important about the content is missing)
Face Validity
The extent to which the items look like they are measuring the construct
Content Validity Example
If a therapist is using a questionnaire to assess depression, the questionnaire should include questions that cover all the symptoms of depression, such as mood, energy levels, sleep patterns, and appetite. It ensures that the test comprehensively covers all relevant aspects of depression
Criterion Validity
Correlation between a test score and some external criterion (e.g., a validated test/measure) – compares it against some other measure or outcome already considered to be valid
Forms of Criterion Validity
-Convergent Validity
-Discriminant Validity
-Concurrent Validity
-Predictive Validity
Convergent Validity
This checks if the test correlates well with other tests that measure the same or similar constructs
Discriminant Validity
This checks if the test does not correlate too strongly with tests that measure different constructs
Concurrent Validity
This checks if the test correlates well with a criterion measure taken at the same time
Predictive Validity
This checks if the test can predict future outcomes or behaviors
Convergent Validity Example
A new depression questionnaire (Designer Depression Inventory) should have similar results compared to an established depression test (Beck Depression Inventory)
Discriminant Validity Example
The depression questionnaire should not have high correlations with a test measuring something different, like delusions
Concurrent Validity Example
If a therapist gives a new depression test (Designers Depression Inventory) and the established depression test (Beck Depression Inventory) at the same time, they should both show similar results
Predictive Validity Example
If a therapist uses an assessment to predict how well a client will respond to a specific therapy in the future, the results should match the actual outcomes
Factors impacting the size of the reliability coefficient
-Restriction of range issues
-Reliability of predictor and criterion measures
-Sample size
-Screening people into dichotomous categories
-Sensitivity and specificity
Restriction of Range Issues
Not representative of a larger population (e.g., if I am looking at depression in college students, it will not be representative of an adult population)
Reliability of Predictor and Criterion Measures
As the reliability for either the predictor or the criterion gets lower, the coefficient is less likely to be significant
Spearman’s Correction for Attenuation
-Correcting for measurement error to strengthen your predictor
-Takes out the effective measurement error for the coefficient
-Reduces the weakening effect of measurement error and specifies the actual correlation between the constructs if they were both perfect and unaffected by error
Sample Size
As the sample size increases, the reliability coefficient needs to be smaller in order to be considered significant (you want it to be 0 to account for as little error as possible)
