Validity Flashcards
(28 cards)
What is validity?
The degree to which evidence and theory support the interpretations of test scores for the proposed uses of a test.
Three faceted concept
content validity, criterion validity, construct validity
Content validity depends on
- content of the test
- internal structure of the test
- psychological processes used in test responses
Importance of validity
- interpretation of behavioural research relies on test validity
- description accuracy depends on ability to measure/manipulate
Validity evidence - test content -
Match between actual content of test and content that should be included
Threats to content validity
- a test includes construct irrelevant content
- construct underrepresentation
Internal structure of the test
The way in which parts of the test are related to each other
Factor-analysis evidence
statistical procedure for evaluating internal structure
response processes
match between psychological processes respondents actually use and should use
direct evidence
obtained via interviews with respondents, discuss variety of issues
Indirect evidence
eye tracking, statistical analysis of responses
Convergent evidence
degree to which test scores correlate with measures of related constructs, to what degree are test scores correlated with other measures they should be?
Discriminant evidence
degree to which test scores are uncorrelated with other measures they should be correlated with
Concurrent evidence
degree test scores are correlated with relevant variables measured at the same time
Predictive evidence
degree test scores correlated with relevant variables at future point in time
4 methods to evaluate convergent and discriminate associations
- some more conceptually complex than others
- some more statistically complex
- some are old, some are new
- some require more explicit predictors than others
Validity generalisation
evaluating validity coefficients across set of studies
Validity generalisation can address important issues such as -
- reveal general level of predictive validity across smaller studies
- reveal degree of variability among smaller studies
- sources of variability among studies
Factors affecting a validity coefficient
- associations between constructs
- random measurement error
- restricted large
- skew and relative proportions
- method variance
- time
Interpreting a validity coefficient
Must decide whether coefficient is large enough to provide compelling evidence of convergent validity, small enough to provide assurance of discriminant validity
Larger a correlation =
more successful making decisions
Statistical significance
when statistically significant, researchers are confident that sample results are representative of population
When evaluating convergent validity evidence -
expect to find statistically significant coefficients
when evaluating discriminant validity evidence -
expect to find nonsignificant validity coefficients
