validity

Question 1

what is validity?

Answer 1

refers to whether or not a test measures what it intends to measure - indicates the usefulness of the test

Question 2

what is the aim of validity?

Answer 2

the ability to make accurate inferences from scores on a test, giving meaning to test scores

Question 3

how are validity and reliability related?

Answer 3

if a test is not valid, the reliability does not matter, if a test is not reliable then it is not valid

Question 4

what is the difference between reliability and validity?

Answer 4

validity tells you how good a test is for a particular situation, reliability tells how trustworthy a score on a test will be.

Question 5

what do reliability and validity refer to?

Answer 5

reliability refers to the consistency of a measure
validity refers to the accuracy of the measure

Question 6

what are the types of validity?

Answer 6

face validity, content validity, criterion validity, construct validity

Question 7

what is face validity?

Answer 7

when a test on the surface (its face, so to speak) seems to measure what it is supposed to measure

Question 8

which form of validity is the least scientific and why?

Answer 8

face validity - the least scientific of all measures of validity as it is just the researcher’s opinion if the items look valid or not

Question 9

what is the main issue with face validity relating to validity?

Answer 9

a test can have good face validity but not really a valid test (doesn’t actually measure what its intended to measure) - test must feel authentic to participants

Question 10

why is face validity important?

Answer 10

test takers are interested in taking it because it is relevant to them - if participants have doubts about the test, it effects the scores, tests with low face validity usually have low reliability

Question 11

how does face validity effect reliability?

Answer 11

tests with low face validity usually have low reliability, also important to have good face validity so that those who would like to use the test think that it will measure what it is said to measure

Question 12

what are the issues with face validity?

Answer 12

doesn’t refer to what is actually being measured, rather what it appears to measure - determined by a review of items, not statistical analysis - insufficient for claiming a test is valid

Question 13

what is content validity?

Answer 13

the degree to which a test measures an intended content area

Question 14

how does content validity relate to the domain sampling model?

Answer 14

do the items on the test make up a representative sample of the attribute the test is supposed to measure?

Question 15

what does content validity aim to do?

Answer 15

ensure correspondence between items on a test and the content domain

Question 16

how is content validity created when developing a measure?

Answer 16

specifying the content areas covered by the phenomenon when developing the construct definition, writing questionnaires or scale items that are relevant to each of the content areas, developing a measure of the construct that includes the best (most representative) items from each content area

Question 17

why is content validity important?

Answer 17

content validity is the core of a test. If you do not get this right, your test is not useful since it wouldn’t measure what it says it measures. It’s important to specify the content areas covered, and writing questions/items that are relevant in these content areas

Question 18

what are the aspects of content validity?

Answer 18

whether the construct is fully represented - if not = construct under-representation, construct irrelevant-variance

Question 19

what is construct under-representation in content validity?

Answer 19

the test does not capture important components of the construct

Question 20

what is construct irrelevant-variance in content validity?

Answer 20

when test scores are influenced by things other than the construct the test is supposed to measure

Question 21

how is content validity established?

Answer 21

judgement by expert judges, use of statistical methods

Question 22

how is content validity established by expert judges?

Answer 22

judges independently examine the items and decide whether each of the items is weakly relevant or strongly relevant to the content domain of the construct - The value would range from 0 to 1, with higher values indicating better content validity.

Question 23

what statistical methods of analysis can be used to establish content validity?

Answer 23

factor analysis to assess whether items said to relate to each content area fit well together statistically

Question 24

what is criterion validity?

Answer 24

how well the test score predicts or estimates the criterion behavior or outcome, now or in the future

Question 25

what is concurrent criterion validity?

Answer 25

refers to the extent to which the test scores can correctly identify the current state of individuals

Question 26

how can concurrent validity be established?

Answer 26

give two tests to participants - one a known test and the other the new test and measuring the new test results against the known test results to see the correlation between scores

Question 27

what is predictive criterion validity?

Answer 27

how well the test performance estimates or predicts current and future performance on some valued measure

Question 28

how can predictive criterion validity be established?

Answer 28

Predictive criterion validity asks if that score on one test can predict how well you will do in the next test or how well you will handle a task. If you correlate the two scores, a high correlation will indicate high predictive validity.

Question 29

what is a construct?

Answer 29

A construct is a postulated, hypothetical attribute. something we think exists, but it is not directly observable or measurable

Question 30

how is construct validity measured?

Answer 30

by looking at the relationship between the construct and other constructs. We ask to what other constructs is this construct similar and to what other constructs is this construct different?

Question 31

what evidence do we look for in the relationship between one construct and another relating to construct validity?

Answer 31

convergent validity and divergent/discriminative validity - For a test to have good validity, it needs to have evidence for both convergent and discriminant validity

Question 32

what is convergent validity?

Answer 32

scores on the test must have high correlations with other tests that measure the same construct

Question 33

what is divergent/discriminative validity?

Answer 33

scores on a test have low correlations with other tests that measure different constructs.

Question 34

what factors effect validity?

Answer 34

reliability: Any form of measurement error can reduce validity. Importantly, you can have reliability without validity. must demonstrate reliability before validity.
social diversity: Tests may not be equally valid for different social/cultural groups.