Chapter 6 Validity

Question 1

what is validity?

Answer 1

validity is the degree to which a test measures what it is supposed to measure

Question 2

what are the three broad categories of validity?

Answer 2

1. Content Validity
2. Criterion-related Validity
3. Construct-related Validity

Question 3

this type of validity is the estimate of the sufficiency of the variable included in the test (test blueprint: syllabus/tos)

Answer 3

content validity

Question 4

this type of validity is an estimate of how well the test fits a hypothesized theoretical framework

Answer 4

construct-related validity

Question 5

this type of validity is an estimate of how well the test measures what it intends to measure at the time the variable is emitted

Answer 5

ecological validity

Question 5

this type of validity is an estimate of how well the test scores relate to a specific standard

Answer 5

criterion-related validity

Question 6

This type of validity is an estimate of how well the test’s appearance fits its intended purpose

How relevant the test items appear to be

(Questions that ask about actions vs Inkblots on The Introversion/Extraversion Test)

Answer 6

face validity

Question 7

for content validity, what is the source of evidence, procedure, and result?

Answer 7

source of evidence: test blueprint

procedure: Subject-matter experts’ review and/or Content Validity Ratio

result: more than 0.75 CVR is acceptable

Question 7

what is a construct?

Answer 7

a construct is an informed scientific idea developed or hypothesized to describe or explain behavior

Question 8

for criterion-related validity, what is the source of evidence, procedure, and result?

Answer 8

source of evidence: relationship with a criterion (standard)

procedure: correlate with a criterion presently available or will be available in the future (concurrent or predictive)

result: correlation is more than 0.60

Question 9

this type of validity is the degree to which a test correlates with other measures of the same or similar constructs

Answer 9

convergent validity

Question 9

what are the two types of construct-related validity?

Answer 9

1. Convergent Validity
2. Divergent (Discriminant Validity)

Question 10

this type of validity is the degree to which a test does not correlate with measures of unrelated constructs

Answer 10

divergent (discriminant) construct validity

Question 10

what are the two types of validity under criterion related validity?

Answer 10

1. Concurrent Validity
2. Predictive Validity

Question 11

this type of validity, under criterion related validity measures the index of the degree to which a test score is related to some criterion obtained at the same time.

Answer 11

Concurrent validity

Question 11

how do you measure the internal construct validity of a test?

Answer 11

check if the test is homogenous, measuring a single construct

Question 12

this type of validity , under criterion-related validity is an index of the degree on which a test scores predicts some criterion

Answer 12

Predictive validity

Question 13

this term refers to how uniform a test is in
measuring a single concept

Answer 13

Homogeneity

Question 13

this term refers to how a test conveys information about the extent to which the factor determines the test score or scores.

Used to test constructs that may be part of the test

Answer 13

Factor loading

Question 14

this type of factor analysis typically entails “estimating, or extracting factors; deciding how many factors to retain; and rotating factors to an interpretable orientation”

Answer 14

Exploratory factor analysis

Question 14

this term is for a class of mathematical procedures designed to identify factors or specific variables that are typically attributes, characteristics, or dimensions on which people may differ.

Answer 14

Factor Analysis

Question 15

this type of factor analysis involves researchers testing the degree to which a hypothetical model (which includes factors) fits the actual data.

Answer 15

Confirmatory factor analysis

Question 16

this term refers to a factor inherent in a test that systematically prevents accurate, impartial measurement

Answer 16

Bias

Question 17

this term refers to the extent to which a test is used in an impartial, just, and equitable way.

Answer 17

Test Fairness

Question 18

this term refers to the specified procedures for administration & scoring typically including normative data

Answer 18

Test Standardization

Question 19

this term refers to the process of developing tests with a targeted defined group as the population for the test is designed

Answer 19

Sampling

Question 20

this type of sampling refers to random selection of the test subjects

Answer 20

Probability Sampling

Question 21

this type of sampling involves non-random selection based on the convenience or criteria, allowing the test practitioners to easily collect the data

Answer 21

Non-probability Sampling

Question 22

what is the difference of norm referenced vs criterion referenced evaluation

Answer 22

in norm-referenced interpretations of test data, a usual area of focus is how an individual performed relative to other people who took the test. in criterion-referenced interpretations of test data, a usual area of focus is the test taker’s performance.

Question 23

this term refers to a test measure’s purpose to measure the particular context involves demonstrating that a test or instrument measures what it's intended to measure

Answer 23

Validation process

Question 24

this type of rating error is also known as generosity error; occurs when a rater tends to score more leniently than warranted, inflating ratings.

Answer 24

Leniency Error

Question 25

this type of rating error refers to an error where the rater is overly harsh, leading to lower ratings regardless of actual performance (e.g., movie critics who consistently give negative reviews)

Answer 25

Severity Error

Question 26

this type of error is characterized by a rater's reluctance to assign extreme ratings, resulting in a clustering of ratings around the middle of the scale

Answer 26

Central Tendency Error

Question 27

this term refers to a cognitive bias where a rater assigns higher ratings to a rate based on a favorable impression or aspect, failing to discriminate among different behaviors or characteristics.

Answer 27

Halo Effect