psychometrics

Question 1

what is validity?

Answer 1

degree the procedure actually measures what it purports to measure

Question 2

what is reliability?

Answer 2

what is measured is done the same way over and over

Question 3

What are 2 types of tests?

Answer 3

norm referenced

standardized

Question 4

Talk about Normative based

Answer 4

• determines where the performance falls within the reference group
• <2 standard deviations= 5% of the population
• more formal
• focus on group similarity

Question 5

Talk about Criterion-Referenced

Answer 5

distinguish different levels of performance standard

• can use raw score to compare to others in their population (not the norm sample)
• defines specific skills (focus on individual differences)

Question 6

What are 3 types of validity?

Answer 6

• Construct: does it measure theoretical construct intended?; relies on indirect evidence & inference; what is theory test was developed?
• Content: 3 factors: appropriateness of type of items, completeness of item sample, way in which items assess the content
• Criterion-related: Concurrent & Predictive
  
  • Concurrent: how current score on 1 instrument can estimate current score on other measure or test in related area
  • Predictive: how current score on 1 instrument can estimate score on criterion measure later

Question 7

Are types of validity interchangeable?

Answer 7

NO

Question 8

What is the keystone of test development?

Answer 8

Construct Validity

Question 9

What is Fidelity?

Answer 9

doing the task as it was meant to do

Question 10

What are 3 types of reliability?

Answer 10

• Interjudge:
• test-retest:
• split-half: (internal consistency): are all the items related to each other

Question 11

What is inter-judge reliability?

Answer 11

• 2 independent judges on occurrence & type of response
• tests needs good definitions to determine observability, definition, & subjectivity
• determined by judges scores matching
• want close to 1.0 (.9 is good)

Question 12

What is point by point reliability?

Answer 12

agreements divided by (agreements +disagreements) x 100

Question 13

What factors would decrease inter-judge reliability?

Answer 13

• incomplete/ambiguous definitions
• training
• practice
• response complexity
• live versus digital recordings

Question 14

What are basal & ceiling?

Answer 14

Basal: starting pt
Ceiling: stopping point

Question 15

Mode? Median? Mean?

Answer 15

Mode= # most often
Median= middle # when in order
Mean= average

Question 16

What is Standard Deviation?

Answer 16

variance from the mean

Question 17

What percentile is 1.5 SD below the mean?

Answer 17

7th percentile

Question 18

What is the Raw score?

Answer 18

actual # you determine from grading test

-don’t need to report this unless its criterion referenced

Question 19

What is the Percentile Rank?

Answer 19

% of scores which fall above/below a particular Raw score

• 50th %ile =mean
• 10th%ile is below -1 standard deviation

Question 20

What is the Standard Score?

Answer 20

converts Raw score into sets of score that have same mean & SD

Question 21

What are Z scores?

Answer 21

similar to normal curve & standard deviation

Question 22

What are T scores?

Answer 22

mean is 50 and SD=10

Question 23

What is the Stanine?

Answer 23

scores distributed into 9 equal segments (percentiles are not equal segments)

Question 24

People in the lowest stanine are in what SD?

Answer 24

usually 2 SD below

Question 25

What is “within the first stanine?”

Answer 25

same as the 7th percentile or

1.5 SD below the mean

Question 26

Age & Grade scores

Answer 26

BAD

lead to gross misinterpretations of performance

Question 27

What is SEM?

Answer 27

Standard Error of the Measurement- (any measure subject to error)
increases our precision that the observed score is reasonably close to the true score

Question 28

What is the Confidence interval?

Answer 28

the range around the observed score

ex: CI of 95% infers that 95/100 x the true score would fall in range

Question 29

How to determine eligibility?

Answer 29

where does the range of scores fall?

Is there enough error on test to determine that person’s score is WNL?

Question 30

What is sensitivity?

Answer 30

is test sensitive enough to capture those with an actual problem? (True Positive) (False Negative shows problems with sensitivity)

Question 31

What is Specificity?

Answer 31

Not over-catching (able to sort out the ones that do not have a disorder (True negative)

Question 32

What are ideal Test requirements?

Answer 32

• Good reliability (retest=.85-.9) & validity
• Good specificity & sensitivity
• Specific criteria for test administration & scoring

Question 33

What are some common errors?

Answer 33

• ignoring cultural makeup of normative sample
• forgetting that tests distort what they are designed to examine
• measuring tx progress with standardized test