Research Methods III Flashcards
(181 cards)
1
Q
Covariance
A
- Reflects the degree to which 2 variables vary together.
- Relationship between two continuous variables in original RAW (unstandardized) scale.
- Scale dependent.
2
Q
Sum of Squares
A
Compute deviations for X & Y and E (cannot be a negative value).
3
Q
Sum of Products
A
Compute product of xy deviations and E (error)
4
Q
Correlation
A
Standardized (z-score) measure of linear relationship between 2 continuous variables.
- Standardized.
- Z-score.
- Scale invariant.
5
Q
Fisher z-test
A
Testing 2 independent sample correlations.
6
Q
Effect of r
A
- Pearson correlation
- Provides a measure of effect size due to being based on standardized scores.
+/- 1 = small effect, +/-3 = medium effect, +/-5 = large effect
7
Q
Regression
A
The statistical technique to produce the best straight line to predict Y.
8
Q
Regression equation
A
Yi = bo + biXi + Ei
9
Q
Yi
A
Dependent or outcome variable, criterion variable
10
Q
Xi
A
Independent variable, predictor variable
11
Q
bi
A
Regression coefficient for the predictor.
Gradient (slope).
12
Q
bo
A
y intercept
value of y when x = 0
13
Q
Ei
A
the errors in prediction based on the regression
14
Q
Assumptions of Regression:
Linearity
A
Based on linear correlations, assumes linear bivariate relationship between each x and y, and also between y and predicted y.
15
Q
Assumptions of regression:
Normality
A
Normally distributed, both univariate and multivariate distributions of residuals.
Y scores are independent and normally distributed (Shapiro-Wilk)
16
Q
Assumptions of regression:
Independence of scores
A
Independence of Y (outcome: DV) scores.
17
Q
Assumptions of regression:
Independence of errors
A
Errors (residuals) from observations should not be correlated with each other (Durbin-Watson test)
18
Q
Assumptions of regression: Minimal multicollinearity
A
Predictors (IVs) should not be highly correlated with each other.
No higher than r = .80 for predictors.
Want Variance Inflation Factor (VIF) to be less than 10.
19
Q
Assumptions of regression:
Homoscedasticity
A
Variance of residuals are uniform for all values of Y (test with Levene’s test) assumed with the sample size is large. Cannot be assumed if sample size is small.
20
Q
Ordinary Least Squares regression (OLS)
A
- Yields values for b-weights (regression coefficients) and the y-intercept that will result in the sum of the squared residuals being at the minimum (smallest).
Best fitting line = smallest total error.
Resulting regression line = least-square error solution.
B-Weights + y-intercept = SS Residuals at minimum
21
Q
Partially standardized regression coefficient
A
Regression coefficient predicting Y from X.
Only standardized on x, not y.
22
Q
The Regression Model & Sum of Squares
A
Squaring each of the deviations and summing across observations yields SS for each source of variability of y.
23
Q
ANOVA to test the Regression Model:
Regression
A
Variability in y that can be explained by the predictor(s) – represents the component of Y that is shared with x1
24
Q
ANOVA to test the Regression Model:
Residual
A
Variability in Y that cannot be explained by the predictor – simply what is ‘left over’ after accounting for X.
25
t-test to test the Regression coefficient
b-cofficients
unstandardized (raw) regression coefficients
26
t-test to test the Regression coefficient:
B(beta) standardized (z-score) regression coefficients
One standard deviation increase in X results in an expected change of beta standard deviation units in Y.
Increase in X, change in Y
27
Suppression
A second predictor variable (X2) that is unrelated to Y (dependent variable) raises the amount of variance explained by the first predictor by eliminating certain irrelevant aspects of the first predictor (X1).
X2 suppresses some of the "error" or "irrelevant" variance in X1.
28
Classical suppression
r = 0, but beta = 0
29
Surprising Suppression
beta > r
Surprising because the Beta values increased from both X1 and X2.
30
Partial Correlation
Correlation between 1 DV and 1 IV variable (Y and X1) with two or more variables (e.g., X2, X3) partialed from BOTH DV and the first IV variables.
31
Semi-partial Correlation
The part correlation has variable 2 ONLY partial out of predictor 1. It is the correlation of Y with that part of X1 which is independent of X2.
32
Coding
Regression framework is flexible.
Categorical or nominal independent variables can be used in Multiple Regression/Correlations.
33
Vectors
g - 1
Represent df of IVs
All coding systems come up with the same correlations, BUT they produce different regression equations.
34
Dummy Coding
Os and 1s
Representation of a variable consisting of g categories by creating g - 1 variables (vectors) for which each g - 1 categories is coded 1 on a single variable while the remaining categories are coded 0 on these variables.
35
Regression = ...
ANOVA
36
Contrast Coding
Identification of specific comparisons of interest & assigning values that enable the treatments to be directly compared.
In a two group design, one group is assigned +1 and the other group is assigned -1.
37
Effects Coding
A method of coding categorical variables in which each group is compared to the weighted or unweighted mean of all the groups.
38
Effects Coding:
Unweighted effects coding
Used when you wat to compare the mean of a particular group with the grand mean, regardless of proportions in population or in sample.
39
Effects coding:
The group coded -1 is known as...
The reference group.
40
Effects coding:
The group coded 1 may be referred to as...
A coded group.
41
bo = ?
Grand mean across all groups.
42
b1 = ?
Difference between the group mean and the grand mean.
43
Dummy Coding b-weight
Indicate differences between conditions.
44
Contrast Coding b-weight
Used to calculate difference between conditions compared.
45
Effect Coding b-weight
Indicates difference between Ya & Yt.
46
Dummy Coding y-intercept
Y-bar when all X's are 0; the mean of the reference group
47
Contrast Coding y-intercept
Y-bar; mean of all the groups
48
Effect Coding y-intercept
Y-bar T; mean of all the groups
49
Ordinal regression
When range is sufficient (i.e., approximately) > 6ish, treat as continuous; if not, trat as nominal.
Only need 1 DS if treated as continuous.
The DF is larger (k -1) if treated as nominal, which can reduce power.
50
Power of regression predictor type from highest to lowest:
Continuous, ordinal, nominal
51
Interaction
The effect of 1 IV on DV changes based on level of another IV.
If each factor has 2 levels (or 2 groups), only one vector is needed to differentiate the 2 factors, in Multiple Regression (MRC).
52
How to code for interaciton
The code for the interaction is simply the multiplication of Vector A and Vector B.
53
Correlation when there is no IV effect
Correlation is different from 0 when there is an IV effect because the means for both treatment groups are different.
54
R-squared for overall regression
Sum all the R-squareds for the main effect of A, the main effect of B and the interaction.
55
R-squared is analogous to Sum of Squares
SS is used for ANOVA and R-squared is analogous to Mean Square.
56
Is it a different or same result in MR (multiple regression) as in ANOVA?
Same!
57
Multicollinearity problem if you don't center
Remove any shared variance between the interaction (A x B) variable & the independent variables that make up the interaction term.
Correlations for the variables X & Z with Y do not change when you center.
58
The individuals predictors' correlations with the interaction....
...does change to 0 when you center.
59
Centering corrects for multicollinearity and...
...avoid accounting for parts of Y more than once.
60
By centering, the regression coefficient of the individuals predictors are...
...the main effect of the predictor.
61
In regression equations without interaction terms...
The y-intercepts are different bu the b-weights are exactly the same.
62
In regression equations with interaction terms...
The y-intercepts and b-weights are different.
63
Interpreting the plots for possible interactions between 2 continuous variables
If lines cross there is an interaction.
64
MR Formula:
Y = bo + b1 x X1 + b2 x X2 +b3 x X1 x X2 + ey
65
b1
Regression coefficient specific to when the value of predictor X2 = 0
66
b2
Regression coefficient specific to when the value of predictor X1 = 0
67
b3
The interaction's predictive effect.
Describes how b1 and b2 change as a function of X2 and X1.
68
The value of b1 changes by b3 units....
For every one unit increase in X2.
69
The value of b2 changes by b3 units...
For every one unit increase in X1.
70
Interpreting the coefficient for the interaction term (b3):
The tobs statistic for b3 provides...
A NHT (null hypothesis test) for X1 & X2.
71
Interpreting the coefficient for the interaction term (b3):
If p(tobs)
Reject Ho & conclude the magnitude of b1 depends on the level of X2 and that b2 depends on X1.
72
Interpreting the coefficient for the interaction term (b3):
If p(tobs) > .05, we...
Fail to reject Ho & conclude the magnitudes of b1 & b3 are constant across all values of X2 & X1 -- usually droped x1 x x2 to improve precision.
73
Simple Slope analysis
Interpret the simple-slope plots to see if the coefficient for the interaction term is positive, 0, or negative.
74
Non-linear (curvilinear) regression
Squaring the predictor
75
Looking at the graphs to see if the quadratic predictor has a positive or negative effect on the regression line:
If the lines are curving down, it is negative, and if it is curving up, it is positive.
76
Proportion & probability of a nominal dependent variable
For a dichotomous variable Y coded [ 0, 1 ]
77
The mean as a proportion for a nominal variable
Can't do multiple linear regression (OLS) with nominal DV because it violates normality and homoscedastic assumption.
78
OLS or linear least squares
A method for estimating the unknown parameters in a linear regression model, based on the smallest sum of the squared errors between the regression line and the raw Y scores.
79
Assumptions that do not need to be met in logistic regression:
- Linearity.
- Homoscedasticity.
- Normality.
80
Assumptions of logistic regression:
Dependent variable should be measured on a dichotomous scale.
Have one or more independent variables (ICs), which can be continuous or discrete.
Independence of observations.
Dependent variable should have mutually exclusive and exhaustive categories.
Independence of errors.
81
Other assumptions of logistic regression:
Little to no multicollinearity.
Linear relationship between any continuous independent variables (IVs) and the logit transformation of the dependent variable.
Large sample sizes.
Needs at least 10 cases per independent variable, some recommended at least 30 cases.
No outliers.
82
Latent Response Variable (y*)
a variable that cannot be directly measured or observed
one approach to conceptualizing Logistic Regression (LogReg)
83
The Logistic Curve
Relates the independent variable, x, to the rolling mean of the DV
84
Estimating the b weights via maximum likelihood
b-coefficient for continuous normal y as in, linear regression were estimated using 'least squares'
no possible for dichotomous Y (logistic model)
Rely on Maximum Likelihood Estimation (MLE)
85
Odds Ratio
Exponential of B
Indicator of the change in odds resulting in a unit of change in the predictor.
86
Better measures of R-squared for Logistic Regression
Cox & Snell's R-squared
common measure of "effect size"; measures how well the logistic regression model fits the data in predicting the dependent variable
87
Better measures of R-squared for Logistic Regression:
Nagelkerke's R-squared
This is an adjusted version of Cox & Snell R-square that adjusts the scale of the statistic to cover the full range from 0 to 1.
Chi-square test of independence to statistically test the logistic regression.
88
Null deviance
Represents the deviance of the model that contains no predictors other than the constant.
89
Residual deviance
Represents the deviance of the model. This deviance should be LESS than the null deviance because the lower value represented BETTER accuracy.
90
Why need a LR with just the y-intercept (Step 0)?
Want to test if against the other models within predictors to see if they are better models.
91
Regression to the Mean (RtM)
A statistical phenomenon that occurs whenever you have a nonrandom sample from a population and two measures that are imperfectly correlated.
92
Characteristics of Regression to the Mean
- Statistical phenomenon, can occur because sample is not random.
- Group phenomenon.
- Happens between any two variables.
- Relative phenomenon.
- You can have regression to the mean occur going up and down.
- The more extreme the sample group, the greater the regression to the mean.
- The less correlated the two variables, the greater the regression to mean.
93
The Problem of RtM in Repeated Measures Design
- In a between-groups scenario TRUE random sampling & assignment ensures that RtM effects are equivalet across conditions.
- Some studies sample 'special' populations and/or non-randomly assign subjects to conditions.
94
Pair-link diagram
The x-axis is suppressed and two X axes are constructed.
95
Galton squeeze diagram
- It is similar to the pair link plot but standardized variables are used & the Y axis represents the means on the post test for each score of Y.
96
The problem of repeated measures t-test to study change...
They don't keep into account baseline.
97
The problem of using change scores to study change...
They don't keep into account baseline.
98
ANCOVA as an alternative to study change
A sequential regression model that examines the treatment effect while controlling for pretest scores. Covary out baseline scores.
In a repeat measures design, the covariate is the pre-test or baseline scores. It examines post-test scores while controlling for pre-test.
99
Lord's paradox
Raises the issue of when it is appropriate to control from baseline status. In three papers, Frederic Lord noted that different results obtain if researchers adjust for pre-existing differences.
Depending on the data analyses you run, you can end up with completely different results.
100
The common problem with t-test & ANCOVA in studying change
What they don't take into account is that within a group, individuals have different baselines and can have different levels of change.
101
The 3 ways to analyze repeated measures data (longitudinal data):
- Change scores
- General Linear Model (GLM; aka repeated measures ANOVA or ANCOVA)
- Mixed Effects Model
102
Regression to the model problem in ANCOVA
This approach attempts to model differences in Ts 'controlling for' a person's T1 score. Ironically, whenever groups are not exactly equivalent at T1, this approach actually introduces RtM.
103
Long or short format for mixed effects modeling?
Long format.
104
Residual covariance
What's left after accounting for average group effects.
105
Advantages of using a Mixed Effects Model:
Missing Data
This default approach to missing data in nearly all statistical packages is Listwise Deletion, which drops any observation with any missing data on any variable involved in the analysis. If the percentage missing is small & the missing data are a random sample of the data set, this is a reasonable approach.
106
Advantages of using a Mixed Effects Model:
Post hoc tests
Because of the way the Sum of Squares are calculated in the multivariate approach, post-hoc tests are not available for repeated measures factors. They are available, however, using the mixed model.
107
Advantages of using a Mixed Effects Model:
Flexibility in treating time as continuous
Depending on the design of the study rather than consider time as four categories, it can be more accurate to treat time as a continuous variable. This allows you to model a regression line for time, rather than estimate four means.
108
Advantages of using a Mixed Effects Model:
A single dependent variable can be used in other analyses.
A study have two-factor (2 x 4) repeated measures design to see if the impact of these two factors on an outcome was mediated by a third variable. Each subject has eight values of the mediator (one for each of the conditions) and eight values on the final outcome.
109
Advantages of using a Mixed Effects Model:
Easier to build into larger mixed models
In a study examining schools and teacher, we can have a cluster of children within teachers. If so, we would need to include teacher as another level in the mixed model. Changing from a 2 to a 3 level model is simple to do if the model is already set up as a mixed model.
110
Fixed effects
Population or Group average effects.
111
Random effects
Deviations (of the individuals in a group) from the population (group) average effects.
112
Interpreting the mixed effects model:
Y-hat i = bo + b1 x1 + uo + u1zi + Ei (standardized s-score formula)
- fixed intercept = the population average of the y-intercept
113
uo
the deviations from the population average of the y-intercept effect
114
ui
the deviations of the population average slope for each of the subjects
115
Maximum Likelihood (ML) Estimation
- RAW ML (full Information Maximum Likelihood; FIML)
- Assumes missing data are Missing at Random
- Generally biased data downward (that can result in negative variances)
116
Best Linear Unbiased Estimator (BLUE)
For the coefficients that you get from the model.
Used to compare nesting, to compare various different structures that we fit to our random effects.
When comparing fixed effects models to each other, we cannot use residual MLs. Need to use Raw ML.
117
Random intercepts
For each cluster, a different intercept is allowed to exist.
uo represents the deviations from the population average intercept.
118
Random slopes
The slope will be something varying within that level of the cluster.
So the variable "u1" is exactly the same as x variable in that ui represents the deviation of the population average slope for each of the subjects.
Random slopes should vary across the subjects.
Slope is allowed to be different between clusters.
119
Nesting
Random slopes are generally nested within a random intercept.
There could be multiple random slopes within an intercept.
Random intercepts can be nested within each other.
120
fixed regression intercept (bo)
the Y score when all predictors are zero
121
fixed effect of being 'at a given time' (b1 x T2_dum)
this is the within subjects factor (change overtime)
122
fixed effect of being in a given condition at a given time (b3 x Ts_dum x MBSR_dum)
This is the interaction between the within subjects factor (time) and between subjects factor (treatment group)
123
estress_it
random person & time-specific error/residual
124
Covariance matrix
A raw score version of a correlation matrix where the diagonal are measuring the relationships between the same variable (time point).
125
Residual covariance matrix
The relationships between variances of time points (random effect variance) after taking out any variance due to fixed effect s (Group Average Main Effects, Interactions, & Y-intercept).
126
Unstructured covariance matrix
Not imposing any constraints on the values.
127
Auto regressive covariance matrix
In the growth study dataset, for example, the response variable of each subject is measured at various ages.
We may suspect that the residual error terms within a subject are correlated.
A reasonable choice of the residual error covariance will therefore be a block diagonal matrix, where each block is a first-order autoregression (ARI) covariance matrix.
128
Reading BIC to determine which MEM model is better
If BIC is smaller, we have improved the model.
129
Examining interactions using graphs:
If lines cross, there is an interaction.
130
Number of vectors needed to code nominal variables in MEM regression
Need g-1 vectors.
131
Slicing by 1 IV to look at the sample effects of another IV
Testing for all possible simple effects of Time at different levels of the Condition.
132
Pairwise Comparisons and problem of Experimentwise alpha error
Pairwise comparison table shows the group comparisons at a particular level of treatment condition. If you try to interpret all of these group comparisons, you will need to worry about experimentwise alpha Type I error.
133
How many vectors are needed for continuous IVs (predictors)?
Need 1 vector for continuous variables.
134
In SPSS, "BY" is for...
Discrete IVs.
135
In SPSS, "WITH" is for...
Continuous variables.
136
Type III Fixed Effects
Testing main effects and interactions.
137
Estimated table
Shows the means of all 3 treatment groups at each time point.
138
Univariate tests
Simple effects table.
139
Using Type III Tests of Fixed Effects to interpret main effects of IVs & the Interaction
If the p value is less than .05, then the IV is significant.
140
Sources of missing data
- Unfinished surveys
- Experimenter-level error (in research design or implementation)
- Participant being unavailable for a certain time or dropping out
- Data entry error
141
At what point does missing data become a problem?
What are the different opinions about this?
In OLS, if data is missing at one time for a participant all of that participant's data must be deleted before analysis.
- Different opinions on how much missing data is alloweable (ex: 5% (Shafer, 1999); 10% (Behnet, 2001))
142
Missing Completely at Random (MCAR)
- Assumption that the probability of missing an observation does not depend on any variables.
- No selection bias.
- Events that lead to the data being missing are independent of observable & unobservable parameters of interest.
- If data is truly MCAR, analysis can be done without bias -- however, data is rarely MCAR.
143
Missing at Random (MAR)
- Assumption that the missing data are unrelated to true data value after accounting for other known characteristics of the subject.
- Missingness is not random but can be accounted for by variables that can be added into analysis.
- Ex: Highly depressed individuals may be more likely to drop out of an experiment or not finish their surveys because they are heavily depressed.
144
Missing Not at Random (MNAR)
- Missing data is due to unfixable selection bias.
- Nonignorable nonresponse.
- Leads to biased results.
145
Using logistic Regression to test MAR
You can use logistic regression with the missing cores dummy coded as the dichotomous criterion or outcome variable (Missing score versus Not Missing score) and possible predictors.
146
MEM/RIML to handle missing responses
Full Information Maximum Likelihood (FIML): Directly estimates parameters using ALL observed data for every case.
147
Pros for FIML
- Requires a single step for imputation & analysis.
- Uses all available data even if some cases are missing data.
- Produces unbiased standard error.
- Can be used with smaller sample (N < 100)
148
Cons for FIML
All variables related to the missing data need to be included in the analysis.
149
EM (expectation maximization) algorithm
2- step iterative process. These 2 steps repeats until results converge (Successive iterations do not show different parameters)
1: Expectation: Uses parameter (initially base don complete-case data) to estimate values for missing data.
2: Maximization: Uses complete-case data& estimated values for missing data to estimate new model parameters.
150
Pros for EM
- Minimize bias in parameters so that larger samples yield less bias.
- Ideal for exploratory & reliability analyses.
151
Cons for EM
- Initial estimates based on list-wise deletion (so it does not use all available data).
- Biased standard errors (but bias reduces as sample size increases)
- Less efficient than FIML for hypothesis testing.
152
"Worst" MNAR scenario in GLM
Gives the worst estimate of the true mean.
153
FIML in MEM is best in...
Estimating the group means in all missing types.
154
Discrete/Ordinal Time
All of our prior statistical models treated the Time (week) variable as a discrete ordinal 'factor.'
155
Individual Growth Curves
Instead of predicting the average group change, the emphasis is now on trajectories of change that vary randomly across persons i.
156
Time as a continuous predictor
Provides a potentially more powerful parameterization.
157
Linear model
How many times points are needed?
How many curves in the model?
The model predicts the Y score (DV) based on a straight line.
Needs 3 time points and no curves.
158
Quadratic model
How many times points are needed?
How many curves in the model?
This model predicts the Y score (DV) based on a line with 1 curve (or bend in the line). Needs 4 time points.
159
Cubic Model
How many times points are needed?
How many curves in the model?
This model predicts the Y score (DV) based on a line with 2 curves. Needs 5 time points.
160
Most things that happen to people...
(e.g. change in memory, change in height, change in anxiety, etc.) do not change constantly at the same rate. Linear is many times not realistic.
161
Multilevel Models/Hierarchical models/Random coefficient models
Measurement occasion or time is a nested factor that is nested within a person. It is multilevel or clustered because the Person (Level 2) is a higher level than the measurement of the DV at different time points (level 1).
162
Nesting
Growth curves are implicitly nested. E.g. Time nested in people.
163
Spaghetti plot
In this graph, all 12 linear equations trajectory are plotted in the same graph. The lines represents linear 'trajectories' for each of the first 12 people shown previous database comparing CBT, MSBR, & both.
164
Boi
average group y-intercept
165
B1i
average group slope
166
"i" in Bo and B1
deviation of individual in y-intercept & slope from the group y-intercept & slope
167
y is "gamma"
fixed effects (intercept & slope) at the Pearson level or level 2
168
Fixed Intercept
Yoo is average score across all groups
169
Fixed Slope
Y10 average regression coefficient
170
Uoi is the random intercept
Represents person-specific y-intercept deviations.
171
U1i or any u that is not uoi is the random slope
represents person-specific slope deviations
172
Residual variance (population mean squared)
A measure of what was not predicted by the MEM model, the deviation from the actual score from the predicted score.
173
Level 1 (Occasion) Random Effects
Reflects the error or deviation of the individual subjects' raw scores from predicted y-scores.
174
Level 2 (Person) Random Effects
Reflects predictors based on individual y-itnercepts & slopes
175
Fixed effects
This reflects the group average y-intercept & regression coefficient (slope)
176
Unconditional linear growth curve ?
Linear growth curve with only Time as IV
177
Residual (population mean squared)
unexplained variance (errors in prediction) at Level 1 (occasion)
178
Level 1 Equation )Quadratic Model)
Time is squared in this equation.
179
Using population mean-squared to comparing linear versus quadratic models
Smaller is better. It means that the MEM is doing a better job of fitting the data.
180
Time-invariant covariates (TICs)
Predicting intercepts & trajectories.
181
Conditional quadratic growth model
Include IVs called Time-Invariant covariates (TICs) other than Time. These IVs try to explain between group effects. They are "invariant" because the TIC code or value is the SAME across the rows of a subject.
For instance, treatment condition (CBT, MBSR, & BOTH) is a TIX, someone in the CDT group has the same code for all time points.
