Chapter 5 - Measurement

Question 1

Types of Measures

Answer 1

Self-report measure
- A statement or series of answers to questions that an individual provides about his or her state, feelings, thoughts, beliefs, past behaviors, and so forth.

Physiological measures
- Any of a set of instruments that convey precise information about an individual’s bodily functions, such as heart rate, skin conductance, skin temperature, cortisol level, palmar sweat, and eye tracking.

Behaviour measures
- The systematic study and evaluation of an individual’s behavior using a wide variety of techniques, including direct observation, interviews, and self-monitoring.

Question 2

Neuroimaging

Answer 2

The use of various technologies to noninvasively study the structures and functions of the brain.

These technologies include:
* magnetic resonance imaging
* functional magnetic resonance imaging
* Diffusion-weighted magnetic resonance imaging
* computed tomography
* positron emission tomography

Question 3

Sustained Attention to Response Task

Answer 3

The Sustained Attention to Response Task is a computer-based go/no-go task that requires participants to withhold behavioral response to a single, infrequent target presented amongst a background of frequent non-targets.

Question 4

Classical Test Theory

Answer 4

The theory that an observed score (e.g., a test result) that is held to represent an underlying attribute may be divided into two quantities:
1. the true value of the underlying attribute
2. the error inherent to the process of obtaining the observed score.

Question 5

True Score

Answer 5

That part of a measurement or score that reflects the actual amount of the attribute possessed by the individual being measured.

Question 6

Measurement Error

Answer 6

Any difference between an observed score and the true score.

Measurement error may arise from flaws in the assessment instrument, mistakes in using the instrument, or random or chance factors.

Question 7

Random Error

Answer 7

Error that is due to chance alone.

Random errors are nonsystematic and occur arbitrarily when unknown or uncontrolled factors affect the variable being measured or the process of measurement.

Such errors are generally assumed to form a normal distribution around a true score.

Question 8

Systematic Error

Answer 8

Error in which the data values obtained from a sample deviate by a fixed amount from the true values within the population.

Systematic errors tend to be consistently positive or negative and may occur as a result of sampling bias or measurement error.

Question 9

Reliability

Answer 9

The trustworthiness or consistency of a measure, that is, the degree to which a test or other measurement instrument is free of random error, yielding the same results across multiple applications to the same sample.

Question 10

Internal Consistency Reliability

Answer 10

The degree of interrelationship or homogeneity among the items on a test, such that they are consistent with one another and measuring the same thing.

Question 11

Inter-Rater Reliability

Answer 11

The extent to which independent evaluators produce similar ratings in judging the same abilities or characteristics in the same target person or object.
- It often is expressed as a correlation coefficient = R

It consistency is high, a researcher can be confident that similarly trained individuals would likely produce similar scores on a target of the same kind.

If consistency is low, there is little confidence that the obtained scores could be produced with a different set of raters.

Question 12

Test-Retest Reliability

Answer 12

A measure of the consistency of results on a test or other assessment instrument over time, given as the correlation of scores between the first and second administrations.

It provides an estimate of the stability of the construct being evaluated.

Question 13

Inter-Method Reliability

Answer 13

A measure of the consistency and freedom from error of a test as indicated by a correlation coefficient obtained from responses to two or more alternate form of the test

Also called alternate-forms or parallel-forms reliability

Question 14

Validity

Answer 14

The degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of conclusions drawn from some form of assessment.

Validity has multiple forms, depending on the research question and on the particular type of inference being made.

Question 15

Construct Validity

Answer 15

The degree to which a test or instrument is capable of measuring a concept, trait, or other theoretical entity.

Question 16

Face Validity

Answer 16

The apparent soundness of a test or measure.

The face validity of an instrument is the extent to which the items or content of the test appear to be appropriate for measuring something, regardless of whether they actually are.

Question 17

Content Validity

Answer 17

The extent to which a test measures a representative sample of the subject matter or behavior under investigation.

A form of construct validity evaluated by comparing the content of the measure to the theoretical definition of the construct, ensuring that all aspects of the construct are measured and no extraneous elements are also measured.

If it looks like a duck, swims like a duck, and quacks like a duck, is it a duck?

Question 18

Predictive Validity

Answer 18

Evidence that a test score or other measurement correlates with a variable that can only be assessed at some point after the test has been administered or the measurement made.

Question 19

Concurrent Validity

Answer 19

The extent to which one measurement is backed up by a related measurement obtained at about the same point in time.

In testing, the validity of results obtained from one test can often be assessed by comparison with a separate but related measurement collected at the same point in time.

Question 20

Convergent Validity

Answer 20

The extent to which responses on a test or instrument exhibit a strong relationship with responses on conceptually similar tests or instruments.

Question 21

Discriminant Validity

Answer 21

The degree to which a test or measure diverges from (i.e., does not correlate with) another measure whose underlying construct is conceptually unrelated to it.

Question 22

Key Points: Reliability and Validity

Answer 22

1. Reliability alone is necessary (but not sufficient) to establish construct validity
2. Construct validity is not necessary to establish reliability
3. Reliability and indicators of construct validity (face, content, predictive, etc.) are both necessary to establish construct validity