Hays, Cha. 5: Measurement Concepts

Question 1

What is reliability?

Answer 1

Looking to see if you get the same results from the same test from the same person during a short amount of time

Question 2

Instrument assessments look for _____, while personality tests look for _____.

Answer 2

growth; stability

Question 3

Why do we need to be careful when assessing children for personality disorders?

Answer 3

They are still developing their personality

Question 4

What is measurement error?

Answer 4

The goal is to reduce the error, but there is going to be some error in your test (known as “error score”)

Question 5

What are the two parts (scores) of each test?

Answer 5

True score & error score

Question 6

(T/F) No score will be perfectly reliable or perfectly without error.

Answer 6

True

Question 7

What are correlation coefficients (aka reliability coefficients)?

Answer 7

When assessing reliability, the statistical measures that determine degree of relationship b/w two factors

Question 8

What is the range of values for correlation coefficients?

Answer 8

-1 to +1

Question 9

(T/F) No relationship will be perfect (score of -1 or +1)

Answer 9

True

Question 10

(T/F) The bigger the number, the stronger the relationship

Answer 10

True

Question 11

(T/F) If the number is “-“, there is a negative effect; as one factor increases, the other factor decreases

Answer 11

True

Question 12

What are the base correlation coefficients for achievement tests an personality tests?

Answer 12

Achievement Test: .85-.90

Personality Test: .50-.60

Question 13

What are the types of reliability?

Answer 13

Test-retest
Split-half
Alternate form
Inter-rater reliability

Question 14

What is test-retest?

Answer 14

Give test on one occasion, give test again later to same group of people and correlation the scores

Question 15

What is alternate form in reliability?

Answer 15

Measuring same construct same way, but with different questions. Give test to a group of people then give the alternate version and correlate scores

Question 16

What is the best way to assess reliability?

Answer 16

Test-retest combined with Alternate form.

Question 17

What is usually used to assess reliability?

Answer 17

Split-half and inter-item (cheap and simple)

Question 18

What is inter-rater reliability?

Answer 18

Whether different raters will have same results on same assessment

Question 19

What is validity?

Answer 19

Does the test measure what it says it’s going to measure? Is the test measuring a construct adequately?

Question 20

Validity Notes

Answer 20

More difficult to assess than reliability

Assessments are usually only good for one of two assessments at most

Question 21

What are the different types of validity?

Answer 21

Validity coefficients
Face validity
Content validity
Criterion related validity
Construct validity
Treatment validity

Question 22

What are validity coefficients?

Answer 22

The statistical measures that determine degree of relationship b/w two factors.
- Usually lower than reliability coefficients.

Question 23

What is face validity?

Answer 23

Does it look like the test is measuring what it is supposed to measure?
- Should increase motivation for test-taker to do well

Question 24

What is content validity?

Answer 24

Is the test assessing the appropriate content for the body of study that a group of students has gone through

• Applies mostly to achievement tests
• A group of experts looks at a group of items to make sure they fit

Question 25

What is criterion related validity?

Answer 25

Comparing scores with performance

Question 26

What are two types of criterion related validity?

Answer 26

• Concurrent validity: Give a new test at the same time as an older one measuring the same thing, then correlating their scores. The results should be equal
• Predictive validity: Prediction of how well a person will do with a certain construct (ex: pilot training, ASVAB)

Question 27

Can we predict low base-rate behaviors (suicide, murder)?

Answer 27

Can be done (Suicide and homicide prediction can be done, but has false-positives because it’s a low base-rate behavior (doesn’t occur very often), so the assessments can’t be used)

Question 28

What is a low base-rate behavior?

Answer 28

A behavior (suicide and murder) that does not occur often.

Question 29

What is a false-positive?

Answer 29

Saying someone is a certain way when they aren’t.

Question 30

What is a false-negative?

Answer 30

Saying someone is not a certain way when they are.

Question 31

What is construct validity?

Answer 31

Are you really measuring what you’re trying to, or you measuring something else?

Question 32

Can you have a reliable test that is not valid?

Answer 32

Yes

Question 33

Can you have a valid test that is not reliable?

Answer 33

No

Question 34

Reliability is _____ the concept of validity.

Answer 34

subsumed under

Question 35

What is treatment validity?

Answer 35

Do these tests and their results make any difference in treatment?
- If the person tested concurs and is motivated to take the test, and the results are shared: then it will aid in treatment

Question 36

What is a response-set (aka response style)?

Answer 36

• When you’re giving an instrument, and the person responds to that instrument in a way that are not what is looked for (through distortion or deception)
• We want people to be honest on answers, but some people unintentionally distort subconsciously (ex: checking all “no” at doctor checklist)

Question 37

What are validity scales used for?

Answer 37

Assessing for distortion

Question 38

What tests use validity scales?

Answer 38

Any large, broad scale personality measure

Question 39

If a person denies any negative behavior, they are probably…

Answer 39

lying. Every one has “chinks in their armor”

Question 40

What will validity scales show?

Answer 40

Distortion - denial of all negative behaviors
Inconsistencies
A lot of blank answers
A lot of “cannot say” answers
“Yes” responses to extremely odd or infrequent behaviors