Chapter 5 Imporving and Assessing the Quality of Behavioral Measurement Flashcards

1
Q

Data

A

primary material to guide and evaluate behavioral work

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

3 indicators of a trustworthy measurement

A

Validity
Accuracy
Reliability

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What are the three elements for validity.

A
  1. directly measuring socially significant target behavior.
  2. Measuring dimension of the target behavior relevant to the question or concerning about the behavior.
  3. Ensuring that the data are representative of the behavior’s occurrence under conditions during times that are most relevant to the concern about the behavior.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

If any of the 3 elements of validity are lacking even if the data seems valid and reliable, what happened to the data?

A

The validity of the resultant data are compromised, even meaningless.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

When does measurement have validity?

A

Measurement has VALIDITY when it yields data that are directly relevant to the phenomenon measured and to the reasons for measuring it.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Validity if threaten when

A

a. indirect measurement
b. measuring the wrong dimension of the target behavior
c. measurement artifacts

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Indirect measurement

A

when what is actually being measured is in some way different from the target behavior of interest.
- secondhand or filtered info

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Measuring the wrong dimension of the target behavior

A

more threatening to validity than indirect measurement.

ex. sticking a ruler to measure water but it does not tell you about temperature.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Measurement artifacts (AKA?)

A

AKA: artifacts

- Giving a misleading picture of the behavior because of the way measurement was conducted.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Causes of Artifacts

A
  1. Discontinuous Measurement: describes any form of measurement in which some instances of the response class of interest may no be detected.
    Ex. time sampling methods
    • poorly scheduled measurement periods
    • insensitive and/or limiting measurement scales: scales that imposes an artificial floor or ceiling on performance.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Accuracy

A

The extent to which the observed value (i.e.,quantitative label) matches the true state or, TRUE VALUE of the event, as it exists in nature. For something to be considered a TRUE VALUE required special or extraordinary precautions that ensure that all possible sources of error have been removed.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

When is measurement accurate?

A

When observed values (numbers obtained by measuring an event) match the true values of the event.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

If measurement is not valid, accuracy is ____.

A

moot

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Accuracy trumps _______.

A

reliability

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Establishing True values

A

Compare observed values to true values.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What happens when true values cannot be established?

A

Researchers must rely on reliability assessment and measure of inter observer agreement to evaluate the quality of their data.

17
Q

Reliability

A

The extent to which a measurement procedure yields the same value when brought into repeated contact with the same state of nature.

  • consistent measurement
  • the closer the values obtained by repeated measurement, the more reliable they are.
18
Q

What does poor reliability mean?

A

Means problem with accuracy.

19
Q

Threat to measurement accuracy and reliability?

A

Human error: biggest threat

20
Q

Factors that Contribute to Human Measurement Error?

A
  1. Poorly designed measurement systems
  2. Inadequate observer training
  3. Expectation about what the data should look like
21
Q

Poorly Designed Measurement Systems

A
  • difficult and cumbersome measurement systems
  • # of participants to observe, number of behaviors recorded, duration of observation period, &/or duration of observation intervals
22
Q

Inadequate Observer Training

A

Observers need explicit and systematic training:

  1. Select observers carefully.
  2. Train observers to a standard of competency
  3. provide ongoing training to minimize observer drift.
23
Q

Observer Drift

A

When observers unknowingly alter the way they apply a measurement system.

24
Q

When does observer drift occur?

A

When observers have a shift in how they interpret the definitions of the target behavior.