Chapter 5: Bivariate correlation research Flashcards
(14 cards)
Bivariate correlational research
Research in which we try to make claims about relationships: we measure multiple variables and focus on relationships between two variables at a time
The variables that are measured (not manipulated!) determine is a study is correlational, not the statistical tests
Two important factors for construct validity
- Reliability: internal, interrater and test-retest reliability
- Evidence in favor of construct validity itself: face, criterion, convergent and divergent validity
Six aspects of statistical validity
- What is the effect size?
- How accurate is the effect estimate?
- Is the effect statistically significant?
- Has the effect been replicated?
- Do outliers influence the effect estimate?
- Is there range restriction?
- Is the relationship linear or not?
What is the effect size?
A large correlation = a large effect size
Large effect sizes are relevant because you have more accurate predictions and less mistakes
Large effect sizes are typically more important, but there are exceptions: correlation of .03 in medicine that treats heart diseases is worth it + small effects can accumulate in certain situations
How accurate is the effect?
Looking at the accuracy effect is done by looking at confidence intervals (CI)
CI: ‘If I were to repeat this study over and over again, in 95% of the cases, the true value/estimate in the population will be in my confidence interval’ → 95% of the interval will contain the true population
The larger the sample, the smaller the CI
If the 95% CI does not contain 0, the effect is statistically significant
If the 95% CI does contain 0, we cannot rule out that the effect is nonsignificant
Is the effect statistically significant?
We use a cut-off p-value of .05: we are willing to take a 5% risk to say there is an effect, when there really isn’t one
We use p-values to know what the chance is that we find an effect of this size or even stronger in our sample, if there would be no effect in the population
What does p<.05 or p>.05 mean?
p < .05 → reject null-hypothesis
p > .05 → effect is statistically significant: likelihood is smaller than 5% that we would find this effect in our sample if there were no effect in the population
Strong effect size (correlation): chance that p-value is very small will increase
Small effect size: chance that p-value is very high will increase
The larger the sample, the higher the likelihood that the p-value will be small and under the cut-off → significant
Has the effect been replicated?
If the same effect has been found in multiple studies, we can be more sure that the results can be trusted
But, replications are not always trustworthy or successful!
Do outliers influence the effect estimate?
Outliers: observations that report values that are quite different from other participants
Can have a big impact in small samples: you might get stronger effects and make inaccurate conclusions
Is there range restriction?
Range restriction: less variability in a variable which affects or influences the effect sizes that you estimated
Reduces the strength of a correlation in a sample
The smaller the variance, the smaller the correlations
Is the relationship linear or not?
Correlation assumes a linear relationship, if relationship is curvilinear we have to look at scatterplot
How to check for third (confounding) variables
Look at the data to know whether there really is a problem of a third variable that could impact the relationship
Moderators
A moderator causes a different relationship between two variables depending on its value
Difference between a moderator and a third variable
If Z is a moderator, the relationship between X and Y will be different for different values of Z
If Z is a third variable, the only reason why we observe a relationship between X and Y is because they are both related to Z
