Reliability and Validity

Question 1

What is test re-test reliability?

Answer 1

Test re-test reliability is the consistency and stability of the responses - across time.

Question 2

What is inter-rater reliability?

Answer 2

The consistency of examiners - across coders

Question 3

What is internal consistency?

Answer 3

Internal consistency is the consistency of the structure. This is the consistency across items and how similar they are to one another.

Question 4

What are reliability coefficients?

Answer 4

-Level of acceptable reliability depending on the construct being measure, the way the test scores will be used, and the method used for estimating reliability
-Refers to the level/percentage of agreement

Question 5

What is content validity?

Answer 5

-Content validity is the representativeness of the items
-Must examine who came up with the questions, how the items were selected, and how representative are the items

Question 6

What is criterion validity?

Answer 6

-Criterion validity is the relationship of scores to external criteria
-Examines test scores and how they correlate to an outside external construct or outcome measure
-Concurrent validity looks at it at the same time
-Predictive validity looks at the future

Question 7

What is construct validity?

Answer 7

-Construct validity is evidence that items measure a theoretical construct, which is demonstrated with internal consistency
-Examines if an assessment is measuring what it is supposed to measure
-Convergent validity examines how well an instrument measures with other measures of another instrument
-Discriminant validity examines inverse relationships. For example: depression with life satisfaction

Question 8

What is a validity coefficient?

Answer 8

-Validity coefficients are used to determine the relationship between a test and a criterion measure
>.50 is very high
.40-.49 is high
.21-.40 is moderate/acceptable
<.20 is low/unacceptable