Chapter 6: Validity

Question 1

this term refers to the judgment or estimate of how well a test measures what it claims to measure within a specific context.

is based on evidence regarding the
appropriateness of inferences drawn from test scores.

Answer 1

validity

Question 2

this term refers to a logical conclusion or deduction made from test scores.

Answer 2

inference

Question 3

this term refers to a correlation coefficient that measures the relationship between test scores and criterion
measures

Answer 3

validity coefficient

Question 3

this term indicates the added explanatory power of an additional predictor variable beyond those already in use.

Answer 3

incremental validity

Question 3

what are the characteristics of a criterion?

Answer 3

Relevant: Pertains to the matter at hand.

Valid: Suitable for its intended use.

Uncontaminated: Avoids bias from predictor measures

Question 3

what are the three broad categories of validity?

Answer 3

1. Content Validity
2. Criterion-related Validity
3. Construct-related Validity

Question 3

this type of validity is the estimate of the sufficiency of the variable included in the test (test blueprint: syllabus/tos)

Answer 3

content validity

Question 3

this type of validity is an estimate of how well the test fits a hypothesized theoretical framework

Answer 3

construct-related validity

Question 3

this type of validity is an estimate of how well the test scores relate to a specific standard

Answer 3

criterion-related validity

Question 3

This type of validity is an estimate of how well the test’s appearance fits its intended purpose

How relevant the test items appear to be

(Questions that ask about actions vs Inkblots on The Introversion/Extraversion Test)

Answer 3

face validity

Question 4

this type of validity is an estimate of how well the test measures what it intends to measure at the time the variable is emitted

Answer 4

ecological validity

Question 4

for content validity, what is the source of evidence, procedure, and result?

Answer 4

source of evidence: test blueprint

procedure: Subject-matter experts’ review and/or Content Validity Ratio

result: more than 0.75 CVR is acceptable

Question 4

for criterion-related validity, what is the source of evidence, procedure, and result?

Answer 4

source of evidence: relationship with a criterion (standard)

procedure: correlate with a criterion presently available or will be available in the future (concurrent or predictive)

result: correlation is more than 0.60

Question 4

what is a construct?

Answer 4

a construct is an informed scientific idea developed or hypothesized to describe or explain behavior

Question 4

what are the two types of construct-related validity?

Answer 4

1. Convergent Validity
2. Divergent (Discriminant Validity)

Question 4

this type of validity is the degree to which a test correlates with other measures of the same or similar constructs

Answer 4

convergent validity

Question 4

this type of validity is the degree to which a test does not correlate with measures of unrelated constructs

Answer 4

divergent (discriminant) validity

Question 5

this term refers to a mathematical procedure used to identify underlying factors or dimensions on which individuals differ.

Answer 5

Factor Analysis

Question 6

this type of factor analysis involves estimating/extracting factors, determining how many to retain, and rotating them for better interpretability

Answer 6

exploratory factor analysis

Question 6

what are the two types of factor analysis?

Answer 6

1. Exploratory Factor Analysis
2. Confirmatory Factor Analysis

Question 7

this type of factor analysis tests how well a hypothetical factor model fits actual data

Answer 7

Confirmatory Factor Analysis (CFA)

Question 8

this term represents the degree to which a factor influences test scores, akin to a metaphorical vehicle carrying varying amounts of abilities or traits

Answer 8

Factor Loading

Question 9

this term refers to any inherent factor in a test that systematically hinders accurate and impartial measurement, indicating systematic variation.

Answer 9

Test Bias

Question 10

what are the types of rating errors?

Answer 10

1. Leniency Errors
2. Severity Error
3. Central Tendency Error

Question 11

this type of error is also known as generosity error occurs when a rater tends to score more leniently than warranted, inflating ratings.

Answer 11

Leniency Error

Question 12

this type of error is an error where the rater is overly harsh, leading to lower ratings regardless of actual performance (e.g., movie critics who consistently give negative reviews

Answer 12

Severity Error

Question 13

what does the ranking procedure do?

Answer 13

A ranking procedure requires raters to measure individuals relative to one another rather than against an absolute scale.

Question 13

this type of error is characterized by a rater's reluctance to assign extreme ratings, resulting in a clustering of ratings around the middle of the scale

Answer 13

Central Tendency Error

Question 14

what is the purpose of a ranking procedure?

Answer 14

This method can help mitigate rating errors like central tendency, leniency, and severity by forcing the rater to prioritize individuals (e.g., ranking first, second, third).

Question 15

this term refers to a cognitive bias where a rater assigns higher ratings to a rate based on a favorable impression or aspect, failing to discriminate among different behaviors or characteristics

Answer 15

Halo Effect

Question 16

this term, in a psychometric context, refers to how impartially, justly, and equitably a test is used in practice

Answer 16

Fairness

Question 17

what is the importance of ensuring fairness?

Answer 17

Ensuring fairness is crucial for the integrity of testing processes and the validity of test results.