Outcome Measures- Reliability and Validity Flashcards
(64 cards)
1
Q
OUTCOME MEASURES
A
OUTCOME MEASURES
2
Q
Outcome measures can be both ________ and _________. List examples of each.
A
Objective
-ROM and length, Temp, Weight, Height, BP, HR, Strength
Subjective
-Pain, Intelligence, Health, Burnout
3
Q
Measurement is the process of assigning _________ to variables to represent quantities of characteristics by rules. They are the actual results of implementing the ____ that indicate the impact on functioning.
A
- numerals
- POC
4
Q
Very few variables are measured _________, most are not directly observable.
A
directly
5
Q
________ as a _________ _________ are often cheaper and easier to measure than ‘true’ endpoints (ie. it is easier to measure a patient’s blood pressure than to use EKG).
A
biomarkers as a surrogate endpoint
6
Q
What is a biomarker?
A
Objective, quantifiable characteristics of biological process
7
Q
What is a surrogate endpoint?
A
A substitute for clinically meaningful endpoint
8
Q
Psychometric properties to consider to make selection of right measures?
A
- Reliability
- Validity
- Scale of measurement
- Self-report vs. Performance-based measure
- Error parameters (minimal detectable change)
- Clinical utility of the measures
9
Q
What are the 2 most important properties to consider before choosing measurement?
A
reliability and validity
10
Q
Reliability = \_\_\_\_\_\_\_\_\_\_ Validity = \_\_\_\_\_\_\_\_\_\_
A
- consistency
- accuracy
11
Q
_________ is an extent to which a measurement is consistent and free from error. It has consistency time after time with as little variation as possible.
A
Reliability
12
Q
Many reliability coefficients are based on measures of correlation. What is correlation?
A
The degree of association between two sets of data
13
Q
Draw a _______ plot to visualize correlation, and compute _______ _________ (r) to quantify it.
A
- scatter
- correlation coefficient
14
Q
What are the 4 major types of reliability?
A
- ) Test-retest reliability
- ) Intrarater reliability
- ) Interrater reliability
- ) Internal consistency
15
Q
What is test-retest reliability?
A
consistency of repeated measurements that are separated in time
16
Q
What is intrarater reliability?
A
consistency of repeated measurements by the same PT practitioner at different times
17
Q
What is interrater reliability?
A
- consistency of repeated measurements by multiple PT practitioners
- important to be good unless the usefulness of that test is limited
18
Q
What is internal consistency?
A
consistency over multiple items or parts in a test, where each part is supposed to measure one, and only one, concept
19
Q
_______ is the degree to which a test is measuring what it is intended to measure.
A
Validity
20
Q
What are the 3 major types of validity?
A
- Face validity
- Content validity
- Construct validity
21
Q
What is face validity?
A
- Does the content of the test appear to be suitable to its aims?
- the weakest form of the validity
- inspection of items for proper selections made by non-experts
22
Q
What is content validity?
A
- Is the test fully representative of what it aims to measure?
- inspection of items for proper selections made by experts
23
Q
What is construct validity?
A
- Does the test measure the concept that it is intended to measure?
- correlation and factor analysis to check
24
Q
- What is the weakest form of validity?
- What is the strongest?
A
- face
- construct
25
What are the 4 scales of measurement in order based on their relative precision of assigned values?
```
Ratio
-distance, age, time, decibals, weight
Interval
-calender years, C or F
Ordinal
-MMT, function, pain
Nominal
-gender, blood type, diagnosis
```
26
- The categorical scale combines _______ and _________.
| - The continuous scale combines _______ and _________.
- nominal and ordinal
| - interval and ratio
27
________ is an ordered system for a series of questions that provide an overall rating to distinguish among people who demonstrate different intensities measured and is developed to measure attitudes.
Scale
28
What are the 3 types of scales and what are they used for?
1. ) Continuous
- used with continuous variables such as age; blood pressure; years of experience
2. ) Categorical (nominal)
- used with categorical variables such as gender; race; diagnosis
3. ) Ordinal
- used with ordinal variables such as pain measured by minimal; moderate; severe or function by independent; minimal assist; moderate assist; maximal assist; dependent
29
List 3 common scales used and what type of scale they are.
- Dichotomous Scale = categorical (nominal) two point scale
- 5-Point Likert Scale = categorical (ordinal) 5 point scale
- Visual Analog Scale (VAS) = continuous scale with verbal anchors at either end of the scale
30
___________ _______ Outcome Measurement is when the patient is required to perform a set of functional tasks.
Performance Based
31
______-___________ Outcome Measurement when the patient is required to complete a questionnaire.
Self-Reported
32
Is the Functional Independence Measure (FEM) a performance based or self-reported outcome measure?
Performance Based
33
Is the Oswestry Low Back Pain Disability Index (ODI) a performance based or self-reported outcome measure?
-Self-Reported
34
______ ________quantify errors in measurements and are not as widely reported as the reliability or validity are.It is important to determine the amount of error associated with a patient’s outcome measure
because it always comes with some imprecision.
Error Parameters
35
What is Standard Error of Measurement (SEM)?
The amount of variability that can be attributed to measurement error.
-SEM = SD * sqrt(1 - r)
36
What is Minimal Detectable Change (MDC)?
Minimal change that a patient must show on the scale to ensure that the observed change is real and not just measurement error.
-MDC = 1.96 * SEM * sqrt(2)
37
Clinical Utility Considerations:
- __________ of the test for application at the level of pathology or health condition, body function or structure, activity, or participation
- precision of the test to accurately measure change
- _________ of the test to the individual’s situation
- acceptability of the test to the individual
- ____ and _____ of administering the test
- appropriateness
- interpretability
- time and cost
38
What are 7 common outcome measures used in PT?
- Timed Up and Go test (TUG)
- Tinetti Balance Test (Tinetti test)
- Berg Balance Scale (BBS)
- Six Minute Walk Test (SMWT)
- Functional Reach Test (FRT)
- Oswestry Low Back Pain Disability Index (ODI)
- Functional Independence Measure (FIM)
39
- The TUG test is a measure of function with correlates to ________ and _____ risk.
- What is the outcome measure of this test?
- balance and fall
| - the time it takes to stand up from chair and walk 3m
40
- The Tinetti Balance Test is a measure of ________ and ______.
- What is the outcome measure of this test?
- balance and gait
| - number of chronic disabilities
41
- The Berg Balance Scale (BBS) is a measure of ________.
| - What is the outcome measure of this test?
- balance
| - total score computed out of maximum 56
42
- The Six Minute Walk Test (6MWT) is a measure of ________ capacity and ____________.
- What is the outcome measure of this test?
- aerobic capacity and endurance
| - distance covered over a time of 6 minutes
43
- The Functional Reach Test (FRT) is a measure of __________.
- What is the outcome measure of this test?
- balance
- measures the distance between the length of an outstretched arm in maximal forward reach from a standing position, while maintaining a fixed BOS
44
- The Oswestry Low Back Pain Disability Index (ODI) is a measure of permanent ____________ disability.
- What is the outcome measure of this?
- functional
| - total score
45
- Functional Independence Measure (FIM) is a measure of individual's physical, psychological, and ______ function.
- What is the outcome measure of this?
- social
| - total score
46
Summary for Outcome Measures:
- ____________ in a clinical research study is the process of assigning numerals to variables to represent quantities of characteristics by rules is the actual results of implementing the plan of care that indicate the impact on functioning.
- Most measurements are not directly observable, and we can measure only a correlate of the actual property, that we name the _________________________.
- What are 7 common outcome measures in PT?
- What are the 6 psychometric properties for a research study?
- Measurement
- indirect nature of measurement
- Timed Up and Go test (TUG), Tinetti Balance Test (Tinetti test), Berg Balance Scale (BBS), Six Minute Walk Test (SMWT) , Functional Reach Test (FRT), Oswestry Low Back Pain Disability Index (ODI), and Functional Independence Measure (FIM)
- reliability, validity, scale of measurement, self-report vs. performance-based measure, minimal detectable change, clinical utility of the measures
47
RELIABILITY AND VALIDITY CONT.
RELIABILITY AND VALIDITY CONT.
48
Reliability does not imply validity but is a _____________ for validity.
prerequisite
49
Reliability is fundamental to all aspects of measurement because with it we can have ___________ in the data we collect and we can draw rational __________ from the data.
- confidence
| - conclusions
50
Many reliability coefficients are based on measures of correlation. What is correlation?
The degree of association between two sets of data
51
- _____________ visualized correlation.
| - _________________ quantifies correlation.
- Scatter Plot
| - Correlation Coefficient (r)
52
Correlation Coefficient (r) is a measure of linear association between two variables. It is a decimal number between __ and __.
-1 and +1
53
A Correlation Coefficient (r) of 0 means what? -1 means what? 1 means what?
```
0 = no correlation
-1 = perfect negative correlation
+1 = perfect positive correlation
```
54
The ______ of (r) indicates the direction of correlation and the ___________ of (r) indicates the strength of the correlation.
- sign
| - absolute value
55
What is Cohen's kappa (k)?
A measure of agreement between two categorical values
56
What is the formula for Cohen's kappa (k)?
k = (Pr(o) – Pr(e)) / (1 – Pr(e))
Where
Pr(o): probability of the relative observed agreement among two examiners
Pr(e): probability of the expected agreement just by chance among two examiners
57
Pr(o) = probability of the relative ___________ agreement among two examiners
Pr(e): probability of the ___________ agreement just by chance among two examiners
- oberved
| - expected
58
Cohen's kappa (k) is a decimal number between 0 and +1 with 0 being ______ agreement and 1 being _______ agreement.
- no
| - perfect
59
Cohen's kappa (k):
- 0-.4 = _______
- .4-.75 = __________
- .75-1.0 = __________
- poor
- fair to good
- excellent
60
What are the 4 major types of reliability?
1. ) Test-retest reliability
2. ) Intrarater reliability
3. ) Interrater reliability
4. ) Internal consistency
61
What are the 3 major types of validity?
1. ) Face validity
2. ) Content validity
3. ) Construct validity
62
_________________ Analysis is a commonly used method to investigate construct validity. It is a statistical procedure used to test how well the measurement represent the number of constructs.
Confirmatory Factor
63
__________ Analysis s a commonly used method to investigate construct validity. It is a statistical procedure used to classify a group of subjects on the basis of a set of measured variables into a number of different groups
Cluster
64
Reliability and Validity Summary:
- Reliability is ___________ time after time, with as little variation as possible.
- Validity is __________ that a test is measuring what it is intended to measure.
- Types of reliability includes test-retest, _________, _________, and internal consistency
- Types of validity includes face, _________, and __________
- Reliability can be estimated using correlation coefficient (r) for two ___________ measurements and Cohen’s kappa (k) for two _____________ measurements
- Internal consistency reliability can be estimated using __________ alpha
- Construct validity can be estimated using correlation, confirmatory factor analysis, and cluster analysis
- consistency
- accuracy
- intrarater, interrater
- content, construct
- continuous, categorical
- Cronbach's
