Week 6 & 7 Flashcards
1
Q
When do you do a non-parametric test?
A
When the basic assumptions for a parametric test are not met
2
Q
Non- parametric statistics are based on…?
A
- Comparisons of ranks of scores
* Comparisons of counts(yes/no) or “signs” of scores
3
Q
Non- parametric statistics are ___ compared to parametric statistics
A
Non- parametric statistics are less powerful compared to parametric statistics
4
Q
What kind of parametric test do you perform when you have 2 independent groups?
A
Unpaired t-test
5
Q
What kind of parametric test do you perform when you have 2 related scores?
A
Paired t-test
6
Q
What kind of parametric test do you perform when you have 3 or more independent groups?
A
One-way analysis of variance (ANOVA) (F)
7
Q
What kind of parametric test do you perform when you have 3 or more related scores?
A
One-way repeated measures analysis of variance (MANOVA)
8
Q
What kind of non-parametric test do you perform when you have 2 independent groups?
A
Mann-Whitney U test
9
Q
What kind of non-parametric test do you perform when you have 2 related scores?
A
- Sign test
- Wilcoxon signed ranks test (T)
10
Q
What kind of non-parametric test do you perform when you have 3 or more independent groups?
A
- Kruskal-Wallis analysis of variance by ranks (H or x^2)
11
Q
What kind of non-parametric test do you perform when you have 3 or more related scores?
A
Friedman two way analysis of variance by ranks
12
Q
True or False
You’re able to perform a non-parametric test on complex designs like a 2 x 3
A
FALSE
Unable to perform on more complex designs (e.g. 2x3)
13
Q
What question is being asked in the comparison based on ranks in a non-parametric t-test?
A
Is the difference in ranks larger than would be expected by chance alone?
14
Q
What question is being asked in the comparison based on signs in a non-parametric t-test?
A
Is the difference in sign frequencies larger than would be expected by chance alone?
15
Q
What type of test do we use when the IV and DV are both on the nominal level?
A
Chi- Square
16
Q
What are you looking at in a chi-square?
A
Are observed frequencies different than expected frequencies
17
Q
What are the 2 types of chi square?
A
- Goodness of fit
* Tests of independence (association)
18
Q
What do you do in the goodness of fit chi square test?
A
• Compare observed frequencies of 1 variable to uniform frequencies of another
19
Q
What is an example of the goodness of fit chi square test?
A
• Eg: flip coin 50 times. Get 15 heads & 35 tails. Is this difference due to chance or a “real” bias?
20
Q
____ chi square test is much more common?
A
Tests of independence (association)
21
Q
What do you do in the tests of independence (association) chi square test?
A
Compare observed frequencies from 1 variable to observed frequencies of another variable
22
Q
What is an example of the tests of independence (association) chi square test?
A
Eg: Is owning a mac laptop related to gender?
23
Q
What is the McNemar test?
A
Requirement of chi-square is that variable levels must be independent (e.g. can’t be “healed” and “unhealed”)
24
Q
___ is the form of a chi square test that is used for 2x2 with correlated sample
A
McNemar test is the form of a chi square test that is used for 2x2 with correlated sample
25
What is a phi coefficient?
A correlation coefficient for 2 nominal variables/ degrees of association for 2x2
26
The phi coefficient is based off the ___
The phi coefficient is based off the *chi-square test*
27
What is the IV level of measurement for a t- test?
Nominal
28
What is the IV level of measurement for an ANOVA?
Nominal
29
What is the IV level of measurement for a non parametric test?
Nominal
30
What is the DV level of measurement for a t- test?
Continuous
31
What is the DV level of measurement for an ANOVA?
Continuous
32
What is the DV level of measurement for a non parametric test?
Ordinal
33
What is the question asked with a t-test?
Difference between means?
34
What is the question asked with an ANOVA?
Difference between means?
35
What is the question asked with a non parametric test?
Ranks different?
36
What is the IV level of measurement for a correlation?
Continuous
37
What is the IV level of measurement for a regression?
Continuous
38
What is the DV level of measurement for a correlation?
Continuous
39
What is the DV level of measurement for a regression?
Continuous
40
What is the question asked with a correlation?
Strength of association?
41
What is the question asked with a regression?
Strength of prediction?
42
What does a correlation have to do with?
A pair of scores and how much they co-vary
43
What does it mean for something to co-vary?
Directly or inversely proportional. When one is high, so is the other and vice versa
44
What are the things that a correlation looks at?
* Do they vary together (covary)?
* How strong is their linear relationship?
* What is the nature of the relationship?
45
A correlation has to be ___
A correlation has to be *linear*
46
What is a correlation coefficient?
A number that quantifies the strength of a linear relationship that can range from -1 to 1
47
What does it mean when a correlation coefficient is closer to 1, whether positive or negative?
Closer to |1.00|, higher strength of relationship
48
What does the sign of the correlation coefficient indicate?
The direction
49
The tighter the grouping of the linear relationship, the ___ the correlation coefficient
The tighter the grouping of the linear relationship, the *higher* the correlation coefficient
50
What does a 0.00- 0.25 coefficient correlation mean?
Little or no relationship
51
What does a 0.26- 0.50 coefficient correlation mean?
Fair relationship
52
What does a 0.51- 0.75 coefficient correlation mean?
Moderate to good
53
What does a 0.75- 1.00 coefficient correlation mean?
Good to excellent
54
What is the coefficient of determination?
• The square of the correlation coefficient
55
What is the coefficient of determination equal to?
The percent of variance in one variable that is explained (or accounted for) by the other variable
56
What is the significance of the coefficient correlation?
To test the null hypothesis
57
What is the null hypothesis as it relates to the coefficient correlation?
The correlation between variable x and variable y is not significantly different from zero.
58
Coefficient correlation is very sensitive to ___
Coefficient correlation is very sensitive to * sample size*
59
What is the most common type of correlation coefficient?
Pearson Product-Moment Correlation Coefficient (r)
60
When is the Pearson Product-Moment Correlation Coefficient applicable?
When both variables continuous (Interval or Ratio scale)
61
What is the Spearman Rank (rho) Correlation Coefficient (rs)?
Non-parametric analog of Pearson r
62
When is the Spearman Rank (rho) Correlation Coefficient (rs) applicable?
When 1 continuous, 1 ordinal variable or 2 ordinal variables
63
When do you use a Point Biserial Correlation (rpb)?
When one variable is dichotomous, and the other variable continuous (interval or ratio)
64
When does a Point Biserial Correlation (rpb) not work?
Doesn’t work with non-dichotomous nominal (e.g Age & Race)
65
Computationally, a Point Biserial Correlation (rpb) is the same as a ___
Computationally, a Point Biserial Correlation (rpb) is the same as a *Pearson’s r*
66
The results of a Point Biserial Correlation (rpb) is the same as ___
The results of a Point Biserial Correlation (rpb) is the same as *a t-test*
67
When do you use a Rank Biserial Correlation (rrb)?
When one variable is dichotomous (nominal), and the other variable is ordinal
68
A Rank Biserial Correlation (rrb) is computationally about the same as ___
A Rank Biserial Correlation (rrb) is computationally about the same as *Spearman Rank*
69
When do you use a Phi coefficient (Φ)?
When both variables dichotomous
70
A Phi coefficient (Φ) is computationally same as ___ (special case)
A Phi coefficient (Φ) is computationally same as *Pearson’s r* (special case)
71
A scatterplot is ___ with a Phi coefficient (Φ)
A scatterplot is *worthless* with a Phi coefficient (Φ)
72
Can a Phi coefficient (Φ) work with a non- dichotomous nominal?
NO
73
A Phi coefficient (Φ) is similar to a ____, but unlike it, a Phi coefficient (Φ) gives gives strength of relationship, while the ___ only gives statistical significance
A Phi coefficient (Φ) is similar to a *chi square test*, but unlike it, a Phi coefficient (Φ) gives gives strength of relationship, while the *chi-square test* only gives statistical significance
74
A correlation does not tell you ___
Does NOT assess differences or agreement
75
How can an extreme outlier affect the interpretation of a correlation?
Can create inflated correlation with only a few extreme data points
76
Can a correlation data be generalized beyond the range of scores in the sample?
Can’t generalize beyond range of scores in sample
77
Low correlation may be due to ___ range
Low correlation may be due to limited range
78
What is reliability?
Extent to which a measurement is consistent and free from error
79
What can a reliable measurement be expected to do?
A reliable measure can be expected to repeat the same score on two different occasions provided that the characteristic of interest does not change
80
Reliability is closely tied to the concept of ___
Reliability is closely tied to the concept of *measurement error*
81
What are the continuous data reliability coefficients?
* Pearson correlation (r)
| * Intraclass correlation coefficient (ICC) (best)
82
What are the discrete/ categorical data reliability coefficients?
* Percent agreement
| * Kappa (best)
83
What are the problems with using a Pearson correlation (r) to quantify reliability?
1. Assesses relationship, not agreement
| 2. Only two raters or occasions could be compared
84
Why do we prefer to use ICCs and Kappa for quantifying reliability?
Both ICCs and kappa give single indicators of reliability that capture strength of relationship plus agreement in a single value
85
____ is stated in terms of variance
*Reliability coefficients* is stated in terms of variance
86
What is the range of a reliability coefficient and what does it mean?
Range 0-1
0 = no reliability, 1 = perfect reliability
87
The more error variability you have, the ____ reliability coefficient will be
The more error variability you have, the *lower* your reliability coefficient will be
88
Reliability coefficient will be bigger, when ___ is larger
Reliability coefficient will be bigger, when *true variance* is larger
89
What is the equation for the reliability/ correlation coefficient?
True score variability divided by true score variability plus error variability
90
What does a high error variability do to correlation coefficient?
It will reduce it
91
What will not having enough true score variability do to correlation coefficient?
It will reduce it
92
What will happens to correlation coefficient with a large true variance?
It will be bigger
93
What are the things that an ICC measures?
Measures degree of relationship (association) and
| agreement simultaneously
94
ICCs give ____ estimate of reliability (can compare different things)
ICCs give *standardized* estimate of reliability
95
ICC is often reported in conjunction with ____
ICC is often reported in conjunction with * Standard error of the measurement (SEM)*
96
ICC is designed for____ data but can be used with ___ data
ICC is designed for *interval/ ratio* data but can be used with *ordinal* data
97
When can can ICC be used with ordinal data?
If intervals “assumed” to be equivalent
98
SEM gives ____ estimate of reliability (i.e. in units
| of measurement)
SEM gives “unstandardized” estimate of reliability (i.e. in units of measurement)
99
The 6 types of ICC dependent on ....?
* Purpose of study
* Design of study
* Type of measurements taken
100
ICC type defined by ___
ICC type defined by *two numbers in parentheses*
101
What does each number in the parenthesis of an ICC type mean?
The first number is the model and the second number is the form. (2, 6) 2 = model, 6 = form
102
How many models of ICC are there?
3
103
What is model 1 of an ICC?
* Each subject measured by a different set of raters; raters “randomly” chosen
* Rarely used in clinical research
104
What is model 2 of an ICC?
• Each subject measured by same raters; raters “randomly” chosen & representative of rater population; results generalizable
105
What is ICC model 2 commonly used for?
Most common for inter-rater reliability or test-retest reliability
106
What is model 3 of an ICC?
• Each subject measured by same rater(s); raters are only ones of interest; results not generalizable
107
What is ICC model 3 commonly used for?
Most common for intra-rater reliability
108
Rank the models of ICC in order from most conservative to least conservative
- Model 1 (most conservative, lowest number)
- Model 2 (neutral)
- Model 3 (least conservative, highest number)
109
When can a model ICC be used for inter rater reliability?
Can be for inter-rater reliability if study raters only ones of interest
110
What does the form/ 2nd number in parenthesis of an ICC represent?
Second number in parentheses represents number of observations used to obtain reliability estimate
111
When is form = 1?
If only one observation per subject per rater (or rating)
112
When is form a number more than 1?
If multiple observations averaged to get single number for analysis, form = number of observations averaged
113
What ICC is best for clinical measures?
• ICC > 0.90
114
What ICC has good reliability?
ICC > 0.75
115
What ICC has poor to moderate reliability?
ICC < 0.75
116
The interpretation of an ICC depends on ____
The interpretation of an ICC depends on *intended use*
117
ICC estimate based on ____ will always be substantially higher than estimate based on ____
ICC estimate based on *average measures* will always be substantially higher
than estimate based on *single measure*
118
What are the characteristics of reliability for categorical scales?
* Based on frequency table
* Agreements on on diagonal
* Disagreements are all others
119
What is percent agreement?
How often the raters agree
120
How do you calculate percent agreement?
Divide number of agreements by total of all possible agreements
121
What is the problem with a percent agreement?
* Does not account for agreement due to chance
| * Tends to overestimate reliability
122
What is the kappa coefficient?
Proportion of agreement
| between raters after chance agreement has been removed
123
On what kind of data is a kappa coefficient used?
Can be used on both nominal and ordinal data
124
What does a weighted kappa do?
Can choose to make “penalty” worse for larger disagreements
125
What can the weight of a weighted kappa be?
Weights can be arbitrary, and
| symmetric or asymmetric
126
A weighted kappa is best for what kind of data?
Best for ordinal data
127
The kappa interpretation depends on ____
The kappa interpretation depends on *the weights used*
128
What does a kappa value of <0.4 mean?
Poor to Fair agreement beyond chance
129
What does a kappa value of 0.4–0.6 mean?
Moderate agreement beyond chance
130
What does a kappa value of 0.6–0.8 mean?
Substantial agreement beyond chance
131
What does a kappa value of 0.8–1.0 mean?
Excellent agreement beyond chance
132
Internal consistency is often used to do what?
Often used to construct and evaluate scale / questionnaires
133
What does internal consistency estimate?
Estimate how well the items that reflect the same construct yield similar results. So, do different questions measure same concept or indicator?
134
What does cronbach's alpha (a) do?
Represents correlation among items and correlation of each individual item with the total score
135
What is recommended that cronbach's alpha be between?
Recommended that cronbach’s alpha be between 0.70 to 0.90
136
Cronbach's alpha can have ___ or ____ on test/questionnaire
Cronbach's alpha can have *dichotomous or multiple-choice responses* on test/questionnaire
137
What can cronbach's alpha (a) help eliminate?
Can help eliminate items from test/questionnaire that are not homogenous to the set or are not contributing unique information
138
What is response stability?
A way to quantify stability of repeated measures over time
139
Response stability is basically the same as ___
Response stability is basically the same as *test-retest reliability*
140
What are the different ways to test response stability?
* SEM: standard error of the measurement
* MDC: minimal detectable difference/change
* CV: coefficient of variation
141
Standard error of measurement is a ___ measure of reliability, while ICC and kappa is a ____ measure of reliability
Standard error of measurement is a *absolute* measure of reliability, while ICC and kappa is a *relative* measure of reliability
142
SEM is in units of _____
SEM is in units of *measurement as variable*
143
What is SEM theoretically?
Standard deviation of the distribution of theoretical multiple measurements
144
An SEM can be used to create a ____
An SEM can be used to create a *95% CI around a measurement*
145
What is the MDC?
Amount of change in a variable that must be achieved to reflect a true change/difference
146
___ is a mathematical multiple of SEM
*MDC* is a mathematical multiple of SEM
147
What is the coefficient of variation (CV)?
A standardized way to measure variability. (SD divided by the mean times 100)
148
What is the coefficient of variation helpful in comparing and why?
Unit-less, so is helpful comparing variability between two distributions on different scales
149
What is an alternate form reliability?
Comparing different methods of testing same phenomenon with different instruments (goniometer vs inclinometer)
150
What analysis or agreement is seen with an alternate form reliability?
- Limit of agreement
| - Bland- altman analysis
151
What is a bland- altman plot?
When you plot the mean of two measures on the x- axis and the difference between the 2 measures on the y- axis, and the center of the plots is a bias
152
What does a tighter range on the bland altman plot mean?
There is more agreement between the two measures
153
When is there no bias on a bland altman plot?
When the line of bias is at 0
154
When is there a consistent bias on a bland altman plot?
When the points on the plot are on one side of the bias line
155
When is there an asymmetrical bias on a bland altman plot?
When the points are split between the two sides of the bias line
