Lesson 2 Flashcards
Psychological Assessment Reliability and Validity (101 cards)
1
Q
What is Psychometric Theory?
A
A framework that includes Classical Test Theory, Generalisability Theory, and Item Response Theory
2
Q
Define Classical Test Theory.
A
A theory positing that an observed score is composed of a true score and an error score
3
Q
What is Generalisability Theory?
A
An approach that examines multiple sources of error variance simultaneously
4
Q
Explain Item Response Theory (IRT).
A
A modern test theory that adds a guessing parameter to item analysis parameters like difficulty and discrimination
5
Q
What are the core characteristics of Psychological Tests?
A
- Measurement of an attribute
- Standardized administration and scoring
- Use of a manual
- Availability of population norms
- Accepted levels of reliability and validity
6
Q
List sources of test score error.
A
- Testing situation
- Tester characteristics
- Test-taker characteristics
7
Q
Differentiate between norm-referenced and criterion-referenced tests.
A
- Norm-referenced: Compares performance to a reference group
- Criterion-referenced: Compares performance to a benchmark or cut-off score
8
Q
What is Reliability in psychometrics?
A
The reproducibility and consistency of measurement
9
Q
Identify the types of Reliability.
A
- Test-retest
- Equivalent forms
- Internal consistency
- Split-half
- Inter-rater
10
Q
What is the acceptable range for reliability coefficients in research?
A
≥ 0.7 is acceptable, with 0.85+ required in clinical practice
11
Q
Define Standard Error of Measurement (SEM).
A
A statistic quantifying the test error around a client’s obtained score
12
Q
How is SEM calculated?
A
SEM = SD of the test * sqrt(1 – reliability coefficient)
13
Q
What does Validity refer to in psychological testing?
A
The extent to which a test measures what it was designed to measure
14
Q
List the types of Validity.
A
- Content validity
- Criterion validity
- Construct validity
15
Q
What is Content Validity?
A
Whether the instrument appears to be a good measure of the concept
16
Q
Define Criterion-Related Validity.
A
Assesses how well a test with a cut-off score categorizes examinees
17
Q
What is Construct Validity?
A
The degree to which a test measures a particular construct
18
Q
True or False: Reliability is sufficient for Validity.
A
False
19
Q
What is the relationship between Reliability and Validity?
A
Reliability is necessary but not sufficient for validity
20
Q
What are Derived Scores?
A
Transformed scores that provide context and meaning, such as standard scores or percentile ranks
21
Q
List factors affecting test score validity.
A
- Test-taking skills
- Anxiety
- Fatigue
- Rapport with the evaluator
- Motivation
- Understanding instructions
- Physical/mental handicaps
- Language skills
22
Q
What is the purpose of statistics in psychological testing?
A
To evaluate chance variations and determine the amount of error in test scores
23
Q
Define standardization in psychological tests.
A
A set of procedures and characteristics that define how a test is administered, scored, and interpreted
24
Q
What is the significance of Item Response Theory in assessments?
A
It optimizes scales and item discriminability by considering parameters like item difficulty and discrimination
25
What is the definition of standardization in psychological testing?
A fundamental set of procedures and characteristics that define how a test is administered, scored, and interpreted.
26
What are the core characteristics of standardized psychological tests?
* Measurement of an attribute or construct
* Standardized administration and scoring
* Use of a manual
* Availability of population norms for interpretation
27
What is the primary purpose of standardization in psychological tests?
Minimizing error in test scores caused by testing situations, tester characteristics, and test-taker characteristics.
28
True or False: Standardization is designed to introduce more unsystematic variation into test scores.
False
29
What is norm-referenced measurement?
A comparison of a child's performance to that of a representative group, called a norm group or standardization sample.
30
What are derived scores in standardized tests?
Transformed scores that have a predetermined mean and standard deviation for interpretation and comparison.
31
What examples of derived scores are commonly used?
* z-scores (M=0, SD=1)
* T-scores (M=50, SD=10)
* IQ scores (typically M=100, SD=15)
32
How does standardization impact the psychometric properties of a test?
It contributes to reliability and validity by minimizing random error.
33
What is reliability in the context of psychological testing?
The consistency of measurements across different administrations or forms of a test.
34
What is the Standard Error of Measurement (SEM)?
An estimate of the error in an obtained score, helping to construct confidence intervals around a score.
35
What does validity refer to in psychological testing?
Whether a test measures what it is supposed to measure and the appropriateness of conclusions drawn from test scores.
36
List the different types of validity.
* Content validity
* Face validity
* Construct validity
* Criterion-related validity
37
What is the purpose of assigning quantitative values in psychological measurement?
To convey meaningful information about people's attributes.
38
What are the four main scales of measurement in psychology?
* Nominal
* Ordinal
* Interval
* Ratio
39
What role do statistics play in psychological measurement?
They help draw conclusions about scores obtained from tests and measures.
40
What is the relationship between confidence intervals and test scores?
Confidence intervals indicate the likely range of a child's true score, acknowledging measurement error.
41
What is the significance of norm-referenced tests?
They require standardized procedures and allow comparison to a representative sample.
42
Fill in the blank: The process of assigning quantitative values to objects or events is called _______.
[measurement]
43
What are some advanced psychometric concepts discussed in the primer?
* Generalizability Theory
* Item Response Theory (IRT)
* Differential Item Functioning (DIF)
* Meta-Analysis
* Factor Analysis
44
True or False: Reliability is a sufficient condition for validity.
False
45
What is one limitation of standardized tests?
Differences in psychometric properties can lead to different results even for tests measuring the same ability.
46
What is the relationship between statistics and drawing conclusions in psychological assessment?
Statistics enable the evaluation of differences and support the interpretation of variability in human characteristics.
47
What does predictive power assess in the context of test validity?
The accuracy of decisions or conclusions made when classifying individuals based on test scores.
48
What are the two types of predictive validity?
Concurrent validity and predictive validity
## Footnote
Concurrent validity relates to a present criterion, while predictive validity relates to a future criterion.
49
What does predictive power assess?
The accuracy of decisions or conclusions made when classifying individuals into categories based on test scores
50
What is construct validity?
The degree to which a test measures a specified psychological trait or construct
51
What type of evidence supports construct validity?
Correlations with related/unrelated measures or factor analysis
52
What are inferential statistics used for?
Drawing inferences about a population based on a sample
53
What do statistical significance tests determine?
If observed differences are likely due to chance
54
What does effect size provide information about?
The magnitude of results, adding practical meaning to conclusions
55
Name a factor that can affect the validity of test scores.
Factors related to the child, evaluator, intervening events, or robustness of the criterion used
56
True or False: No instrument is perfectly reliable or valid for all purposes.
True
57
What is the significance of estimating error in psychological measurement?
It acknowledges the reality of imperfect scores and helps make appropriate interpretations
58
According to classical test theory, what is an observed test score composed of?
A true score and an error score
59
What does the Standard Error of Measurement (SEM) reflect?
The reliability of a test
60
What does a confidence interval indicate?
A range of scores that likely includes the child's true score
61
What is Generalizability Theory used for?
Examining multiple sources of error variance in measurement
62
What is the highest level of measurement scale?
Ratio Measurement Scale
63
List the four main scales of measurement commonly used in psychology.
* Nominal Measurement Scale
* Ordinal Measurement Scale
* Interval Measurement Scale
* Ratio Measurement Scale
64
What type of statistics are used for analyzing data from nominal and ordinal scales?
Nonparametric statistics
65
What are derived scores?
Transformations of raw scores to provide context relative to a norm group
66
What are the three most commonly used measures of central tendency?
* Mean
* Median
* Mode
67
What is the mean?
The arithmetic average of all scores
68
What is the median?
The middle point in a set of scores arranged in order of magnitude
69
What is the mode?
The score that occurs most frequently in a set of scores
70
What are the three most common measures of dispersion?
* Range
* Variance
* Standard Deviation
71
What does the range measure?
The difference between the highest and lowest scores
72
What does the variance measure?
The amount of variability of scores around the mean
73
What is the standard deviation?
The average distance of each score from the mean
74
What is a normal curve?
A symmetrical bell-shaped frequency distribution
75
What do standard scores approximate?
Interval scales
76
What is a key limitation of percentile ranks?
The units along the distribution are not equal
77
What are the key components of descriptive statistics?
* Measures of central tendency
* Measures of dispersion
* Normal curve
* Derived scores
78
What is the purpose of the normal curve in psychological measurement?
It is useful for comparing an individual's score to a group and for understanding derived scores like percentile ranks and standard scores.
79
What are derived scores?
Transformed raw scores that provide more meaning and context, often relative to a norm group.
80
What are standard scores?
Transformed raw scores with a predetermined mean and standard deviation, preferred for statistical analysis.
81
What is a percentile rank?
Indicates an individual's position relative to a sample, showing the percentage of scores at or below a given score.
82
True or False: Percentile ranks can be used in standard statistical tests like addition or subtraction.
False
83
What is norm-referenced measurement?
A method comparing an individual's performance on a test with a representative group, referred to as a norm group.
84
What is the fundamental idea behind norm-referenced measurement?
A raw score on a psychological test is often not meaningful without context from a relevant group.
85
What is a norm group?
A representative group used for comparison in norm-referenced measurement.
86
What qualities make a good norm group?
Representativeness, size, relevance, and currency.
87
What is the guideline for the size of a norm group?
At least 100 individuals per relevant age group.
88
What are age equivalents or grade equivalents?
Scores representing the average raw score obtained by children at a specific age or grade.
89
What is the Flynn Effect?
Generational changes in test scores that may affect the relevance of norms.
90
What is reliability in the context of psychological measurement?
The consistency of measurements; a reliable test provides consistent results.
91
What does classical test theory state about a test score?
A test score is composed of a true score and an error score.
92
What is a reliability coefficient?
A value representing the degree of consistency in test scores, ranging from .00 to 1.00.
93
What is internal consistency reliability?
Measures the uniformity or homogeneity of items throughout a test.
94
What is test-retest reliability?
Correlating scores from the same test given on two different occasions.
95
What is alternate-forms reliability?
Determined by administering two different but similar forms of a test to the same group.
96
What is interrater reliability?
The degree of agreement among different raters or scorers.
97
What factors can affect test reliability?
* Test length
* Homogeneity of items
* Test-retest interval
* Variability of scores
* Guessing
* Variation in the test situation
* Sample size
98
What is the Standard Error of Measurement (SEM)?
Estimates the amount of error inherent in an obtained score.
99
How is the SEM related to reliability?
Lower reliability means a higher SEM, and vice versa.
100
What are confidence intervals?
A range that likely includes the child's true score, constructed using the SEM.
101
What is Generalizability Theory?
A modern approach that examines multiple sources of error variance simultaneously.
