final Flashcards
1
Q
face validity
A
does a test appear to measure what it was designed to measure; lay-person judgement
2
Q
how do content & face validity differ?
A
content involves systematic and technical analysis
face is more superficial
3
Q
criterion validity
A
the extent to which a measure agrees with a gold standard; whether it matches a measure of some attribute or outcome that is of primary interest (criterion)
4
Q
types of studies: criterion validity
A
predictive vs concurrent
5
Q
predictive studies
A
take the test today and test the criterion some time down the road
6
Q
drawbacks to predictive validity studies
A
time, money, issues from time lag
7
Q
concurrent studies
A
test and criterion done at the same time
8
Q
when should you use predictive vs concurrent studies?
A
if goal is prediction -> predictive
if goal is to determine current status -> concurrent
9
Q
criterion contamination
A
when criterion measures more dimensions than by those by test
do scores on the predictor influence criterion scores?
10
Q
techniques for interpreting validity coefficients
A
(1) sig level; did not occur by chance (p value)
2) coefficient of determination (R^2
11
Q
what if your validity coefficient is small?
A
if a test provides info that helps predict criterion performance better than any other existing predictor the test may be useful even if coefficient is relatively small
12
Q
linear regression
A
a mathematical procedure that allows us to predict values on one variable if we know values on the other
13
Q
standard error of estimate
A
a stat that reflects the average amount of error in our prediction and that allows us to make confidence statement
14
Q
decision theory models
A
when tests are used for making decision such as personnel selection; factors others than the correlation between test and criterion are important
15
Q
decision theory models: selection ratio
A
proportion of applicants needed to fill position
16
Q
decision theory models: base rate
A
proportion of applicants who can be successful candidates
17
Q
model sensitivity
A
metric that evaluates ability to predict true positives of each available category
A/A+C
A= true positive C= false negative
18
Q
model specificity
A
metric that evaluates ability to predict true negatives of each available category
D/B+D
B= false positive
D=true negative
19
Q
evaluating validity coefficients
A
- look for changes in the cause of relationships
- what does the criterion mean?
- review the subject pop of validation study
- be sure sample size was adequate
- never confuse criterion with predictor
- check for restricted range on both predictor and criterion
- review evidence for validity generalization
- consider differential prediction
20
Q
construct validity
A
extent to which evidence can be provided that test measures a theoretical construct
21
Q
Campbell & Fiske’s types of validity evidence
A
convergent and discriminant
22
Q
types of convergent evidence
A
(1) does test measure same thing as other tests used for same purpose
(2) does test correlate with specific variables that we can expect if it is doing its job
23
Q
validation study
A
two or more constructs measured in two or more ways
24
Q
what can validation studies tell us?
A
convergent and discriminate validity
homogenity and unidimensionality
25
evidence of validity based on response process
involves an analysis of the fit between the performance and actions the test takes actually engage in and the construct being assessed
e.g., interview, behavioural indicators (RT, eye gaze)
26
evidence based on consequences of testing
were the intended benefits of testing achieved?
27
ways of getting evidence of validity
(1) test content
(2) relations to other variables (criterion)
(3) internal structure
(4) response processes
(5) consequences of testing
28
factor analysis
any of several stat methods describing the interrelationships of a set of variables by stats deriving new variables, called factors, that are fewer in number than the original set of variables
29
types of factor analysis
exploratory and confirmatory
30
if alpha is lower than expected, there might be ______ and you might want to do _____
heterogeneity
factor analysis
31
steps in factor analysis
(1) extraction (how many groups?)
| 2) rotation (average correlation between items and factor itself
32
purposes of assessment in education
- how well is a student learning?
- assess whether class, grade, school, district, region is learning content
- method to detect learning problems
- method for identifying giftedness
- determine if child is ready to move to next level
- assess teacher effectiveness
- determine readiness/placement in college, grad school, professional school
- credential exams
33
achievement test
assess learned information; evaluate the effects of a KNOWN or controlled set of experiences
34
what type of validity procedures does achievement testing rely on?
heavily on content validation procedures
35
aptitude test
assess ability to learn something; evaluate the effects of UNKNOWN or uncontrolled experiences
36
what type of validity procedures does aptitude testing rely on?
heavily on predictive criterion validation procedures
37
goal of classroom testing
measure the extent to which students have learned the facts, concepts, procedures, and skills that have been taught
38
effective classroom tests
students who have learned more will obtain higher scores and students who have learned less will obtain lower scores. to be an effective test, a test must consist of effective items
39
types of classroom achievement tests
constructed and selected
40
Bloom's taxonomy: levels of understanding
(1) knowledge
(2) comprehension
(3) application
(4) analysis
(5) synthesis/create (switched with 6?)
(6) evaluation
41
item difficulty index
right v wrong questions: percentage or proportion of test takers who correctly answer the item
42
item difficulty index: too hard
.0 -.2
43
item difficulty index: too easy
.9 - 1
44
item difficulty indeces are:
sample dependent and after the fact
45
on constructed response tests with two options, what is the optimal mean p value?
.50 (about half the class gets it right)
46
item discrimination: right and wrong Qs
Pt - Pb
47
item discrimination: good discriminatory
lower % of bottom quarter of class got it correct than top quarter of class
48
item discrimination: bad discriminator
bottom and top quarter of class did equally well on question
49
Examples of achievement tests
(1) Wechsler individual achievement test
(2) standford achievement test
(3) Iowa test of basic skills
(4) metropolitan achievement test
50
Wechsler individual achievement test (WAIT)
z-scores, percentile ranks, stanines
norms for grades and age
all ages (above 4)
45 min - 2 hours
- longer for adults than kids
gifted? learning difficulties?
high reliability
51
Stanford Achievement test
individual test
1923
K-12
math, writing expression, understanding of patterns, reading comprehension
high reliability
evidence for construct validity
52
Iowa test of basic skills
general achievement tests
K-8?
better for lower end of distributions?
shorter than others
53
metropolitan achievement test
classified as achievement test, but has some aptitude components
54
examples of diagnostic tests
(1) wide range achievement test 4 (the rat)
(2) peabody individual achievement test
(3) woodcock reading mastery test
(4) kaufman test of educational achievement
(5) canada quick individual achievement test
(6) canada french immersion achievement test (C-FIAT)
55
wide range achievement test 4 (the rat)
diagnostic test
basic academic skills
good for 5-98
individual admin
longer time frame for older people
56
readiness tests
intended to assess a child's readiness to enter school or move forward
57
issues with readiness tests
(1) children change rapidly
(2) predictive ability is weak
(3) cultural/language biases
58
range rule
standard deviation should be around: (max response-min response)/4
59
examples of aptitude tests: cognitive ability
(1) otis-lennon school ability test
(2) cogAT
(3) SAT-I
(4) ACT
(5) GRE; GMAT; LSAT; MCAT
60
issues with grad school tests
don't predict success and differentially predict for different groups
61
advantages and disadvantages of intelligence testing
advantages: helps identify/define problem
disadvantages: cultural bias, limited info
62
three research traditions
(1) psychometric, (2) information processing, (3) cognitive
63
binet: intelligence
tendency to take and maintain a definite direction, the capacity to make adaptations for the purpose of attaining a desired end and the power of auto-criticism
64
binet: principles of test construction
(1) age differentiation
| (2) general mental ability
65
binet's age differentiation
we should be able to distinguish between people (especially) children of different ages
IQ = MA/CA * 100
max mental age was 19.5 (problem)
66
routing procedure
start test based on chronological age, administrator moves to more challenging items as appropriate
67
Standford Binet - 5
intelligence test
appropriate for a broad range of 2 to 85+ years, providing one assessment for all ages (recommend waiting until school age)
provides comprehensive coverage of five factors of cog ability
(1) fluid reasoning
(2) knowledge
(3) quantitative processing
(4) visual-spatial processing
(5) working memory
assessed verbally and non-verbally
scores: full scale IQ, verbal IQ, nonverbal IQ, routing score (start point), individual scores for each scale (verbal and non-verbal)
68
goodenough-harris drawing test (G-HDT)
non-verbal intelligence test
group or individually administered
standardized
reliability ranges in high .60's to low .90s
69
wechsler: intelligence
aggregate or global capacity of the individual to act purposefully, to think rationally, and to deal effectively with his environment
wanted to focus more on adults (unlike Binet)
70
factors that influence performance on intelligence tests
(1) general intelligence, (2) general, (3) specific, (4) influencing factors
71
performance on intelligence tests: general
comprehend, follow direction, respond verbally, understand english
72
performance on intelligence tests: specific
concentration, memory, reasoning
73
performance on intelligence tests: influencing factors (not measured directly)
interests, occupation, confidence, arithmetic skills/knowledge
74
differences between Binet and Wechsler
point scale concept
inclusion of performance scale
75
challenging age differentiation of Binet
IQ = attained or actual score/ expected mean score for age
doesn't max out like binet
76
WAIS-IV
qualification level: C
completion time; 60-90 min for core subtests
ages 16-90
IQ mean = 100; SD = 15
full scale IQ, 4 indices, individual subtests (e.g., arithmetic) with intellective and non-intellective components
pattern analysis
strengths and weaknesses
normative sample = 2200 (US)
high reliability and good evidence of validity
77
raven progressive matrices
one of the best known and most popular
can be administered to group or individuals
from 5 years of age to elderly
used throughout the world
respectable reliability coefficients: high .70 - .90
last revisions to the manual 1998 with impressive set of norms
has been tested with various cultural groups shown to historically score lower on binet and wechsler scales
78
culture fair intelligence tests & an example
one purpose of nonverbal and performance tests is to remove factors related to cultural influences that often disadvantage test takers’ performance
RPM (Raven’s progressive matrices) comes close to being culture fair
IPAT culture fair intelligence test
catell pencil and paper (fluid intelligence in children)
79
Gardner theory of multiple intelligences
intelligence is not unitary; it is the ability to solve problems or to create products that are valued within one or more cultural settings
80
Gardner's types of intelligences
(1) linguistic
(2) logical-mathematical
(3) spatial intelligence
(4) bodily-kinesthetic
(5) musical intelligence
(6) interpersonal
(7) intrapersonal
(8) naturalistic
81
who developed idea of emotional intelligence (EQ)?
Peter Salovey (Yale); followed up by Goleman
82
clusters within emotional intelligence
(1) abstract
(2) concrete
(3) social
83
emotional intelligence has its roots in ____
social intelligence
84
EQ includes:
(1) being aware of one’s own emotions
(2) able to manage one’s own emotions
(3) sensitive to the emotions of others
(4) able to respond to and negotiate with other people emotionally
(5) use one’s own emotions to motivate oneself
85
EQ allows us to:
regulate emotions and problem solve
86
what did Goleman include in EQ?
conscientiousness, self-confidence, optimism, communication, leadership and initiative
87
Examples of reasons for neuropsychological testing
dementia, alzheimers, concussion, brain injury, ALS, parkinson's, stroke, epilepsy, brain tumour, infection
88
quick and dirty assessment neuropsych tool
glasgow coma scale (GCS)
89
neuropsychological testing
application of a set of standardized procedures designed to assess and quantify brain function as expressed in over beh
leads to additional inferences regarding the covert processes of the brain
90
difference between neuropsych testing and general intelligence measures
neuropsych tests tend to be more highly specific in what they measure
91
components of neuropsych testing
1. all (or at least a sig majority) of a patient’s relevant cog skills or higher order info processing skills should be assessed
2. testing should sample the relative efficiency of the right and left hemispheres of the brain
3. testing should sample anterior and posterior regions of cortical function (posterior mostly receptive)
4. testing should determine the presence of specific deficits
5. should determine the acuteness versus the chronicity of any problems or weaknesses
6. testing should locate intact complex functional systems
7. testing should assess affect, personality, and behaviour
8. test results should be presented in ways that are useful in a school or work environment, to acute care or intensive rehabilitation facilities or to physicians
92
two conceptual approaches: neuropsych testing
(1) fixed battery approach
| (2) non-fixed
93
example of a fixed battery approach to neuropsych testing
halstead-reitan neuropsych test battery
focuses on key behavioural correlates of brain function
94
non-fixed battery approach to neuropsych testing
use of a flexible combo of traditional psych and educational tests
e.g., boston process approach
can include qualitative stuff
95
conceptual model of brain-behaviour relationships
1. sensory input
2. attention and concentration
3. learning and memory
4. language
5. spatial and manipulatory ability
6. executive functions (logic, concepts, reasoning, planning, flexibility)
7. motor output
96
example of motor function tests
finger tapping
grip strength
grooved pegboard
97
four factors of mental processing
(1) focus execute, (2) sustain, (3) encode, (4) shift
98
advantages of interviews
- get unique information
- participants can elaborate
- personal and meaningful experience
- report and relationship building
- rich info; detail
99
disadvantages of interviews
- harder to anlayze
- possible discomfort of participant
- not honest, not best performance
- time and resources
- introduction to bias
- individualized/subjective
- limited generalizability
100
types of interviews
(1) structured (highly)
(2) guided/semi-structured
(3) non-directive or unguided
101
initial intake interview
- demographic data
- reason for referral
- past medical history
- present med condition
- familial medical history
- past psych history
- past history with medical or psych professionals
- current psych conditions
102
potential biases in interviews
(1) confirmation bias
(2) self-fulfilling prophecy
(3) ethnocentrism
103
ineffective interviewing
judgmental and evaluative statements, probing questions, false reassurance
104
effective studying
attitude is warm and authentic, open-ended questions, measuring understanding
105
interviews: measuring understanding
levels 1-5 ?
106
sources of error in interviews
```
interview validity
interview reliability (length of session)
```
107
personality
an individual's unique constellation of psych traits that is relatively stable over time
108
personaltiy traits
distinguishable, relatively enduring ways in which one individual varies from another
109
personality types
a constellation of traits
| continuum thinking is in contrast to this
110
personality assessment methods
(1) objective measures, (2) projective measures, (3) behaivoural assessment
111
MMPI
purpose: to aid in diagnosis of psychopathology for adults 14 years and older
developed for abnormal personality
566 true/false items
originally criterion keyed items
112
criterion keyed
way of developing items by how well they discriminate between different groups (e.g., psych pops vs non-psych pop)
113
validity (Messick)
“an integrated eval judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and actions based on test scores of other modes of assessment”
the appropriateness or accuracy fo the interpretation of test scores
114
threats to validity
(1) construct (internal) underrepresentation, (2) construct-irrelevant variance (external), (3) examinee characteristics, (4) test admin and scoring, (5) instruction and coaching
115
construct underrepresentation
not all aspects of construct are represented
116
relationship between reliability and validity
reliability is necessary but not sufficient for validity
reliability restricts validity coefficients
\sqrt{rel}= max validity coeff
