Research Methods 1 Flashcards
(82 cards)
1
Q
What are the assumptions of a one-way ANOVA?
A
- <b>Normally distributed data </b>- Most observations should be symmetrical around the mean.<div>- <b>Homogenity of variance</b> - All conditions should have similar variance</div><div>- <b>Independence of observations</b></div><div>- <b>Interval or ratio measurement </b>- Interval e.g. temperature, ratio e.g. weight, has an absolute zero</div>
2
Q
What should you do if the parametric assumption in a one-way ANOVA is not satisfied?
A
Non-parametric equilvalent<div>For between-participants design = kruskal-wallis</div><div>For within-participants design = Friedman</div>
3
Q
How do you test the assumption of homogenity of variance in ANOVA?
A
Levene’s test
4
Q
Describe the general idea of a one-way ANOVA
A
Compare estimates of variance between groups.<div><br></br></div><div>
Examine differences between groups.
Only have one dependent variable.
When have one IV do a one-way ANOVA
When two IV’s do a two-way ANOVA</div>
5
Q
What does an F ratio of 1 indicate?
A
No effect on the dependent variable being in different groups. No difference between groups
6
Q
What does an F-Ratio less than 1 indicate?
A
There is more variability between the individuals than groups. Groups have no association with scores on the DV.
7
Q
When can normality be assumed in a one-way ANOVA?
A
If Komogorov-Smirnov or Shapiro-Wilk tests are <b>non-significant</b>
8
Q
What is the purpose of a post-hoc test?
A
Tells you which conditions differ from each other. Where the differences lie.
9
Q
What are the parametric assumptions of a two-way between ANOVA?
A
- <b>Normality</b> - Normal distribution. Use Kolmogorov-Smirnov or Shapiro-Wilk if not satisfied.<div>- <b>Homogenity of variance </b>- Data from populations with equilvalent variance. Use Levene’s test</div><div>- Independence of observations</div><div>- Interval or ratio data</div>
10
Q
What does an effect size of 0.01 show?
A
Small effect
11
Q
What does an effect size of 0.06 show?
A
Medium effect
12
Q
What does an effect size of 0.14 show?
A
Large effect
13
Q
How many F-Ratios do you calculate in a 2-way between subjects ANOVA?
A
3
14
Q
A factor has 2 levels is it necessary to carry out a post-hoc test?
A
It is unecessary as it has 2 levels
15
Q
What is a main effect?
A
Effects of each independent variable on the dependent variable
16
Q
What is an interaction effect?
A
The effect of the combination of the 2 independent variables on the DV
Provides more info than any single main effects. If not significant then the main effects are important.
17
Q
What is a factorial ANOVA?
A
Used when there is more than 1 independent variable.<div><br></br></div><div>Calculates main effects and interaction effects</div>
18
Q
When is there an interaction in an ANOVA?
A
When the pattern across one factor differs at different levels of the other factor
19
Q
What are the advantages of within-participants design
A
Reduced error which increases power
20
Q
What are the disadvantages of within-participants design?
A
- Practice effects - People have already done task, so may be better<div>- Carry over effects</div><div>- Needs to be built into the design of the experiment</div>
21
Q
What is a type 1 error?
A
Rejecting a null hypothesis when it is true. There are no differences.
22
Q
What is a type II error?
A
Accepting a null hypothesis when it is false.
Failing to observe a difference when there is one.
Reducing this is a better way of increasing power compared to type 1.
23
Q
What is the power of a statistical test and what does it depend on?
A
The probability of correctly rejecting a false null hypothesis.
Want to increase power as it comes with increased reliability
Depends on
- Type 1 error
- Type II error
- Sample size
- Effect size
- Experimental design
- Choice of statistical test
24
Q
What are the assumptions of a within-participants ANOVA?
A
- <b>Normality</b><div><b>- Interval or ratio data</b></div><div><b>- Sphericity </b>- This replaces homogeneity of variance About assuming that the relationship between scores in pairs of treatment conditions is similar..</div><div>- No independence of observations as there won’t be independence due to having the same people.</div>
25
How do you test for sphericity?
Mauchly test
Only appropriate if have 3 levels as if have 2 then nothing to compare it to.
26
What do you do when Mauchly's test is significant?
Go ahead with ANOVA as is robust
or
Use Greenhouse Geisser or Huyhn-Feldt correction factor with ANOVA.
27
How can a type II error be reduced?
Minimising the overlap between the two distributions
Leads to a reduction in type II error which increases power.
Want to create conditions of low variability as it means minimising error.
28
How can error be reduced?
- Larger sample size: Leads to more valid results and minimises sampling error
- Experimental error: Set up experiment properly, eliminate poor methodology or tech.
- Experimental design - Within-subjects design reduces individual differences, increasing the error
This all increases power
29
Explain what effect size is
Explains the pratical importance of the results. Tell you how to set-up experiment.
Partial eta squared (np2)
Larger sample size = More separate means, more likely to have increased statistical power
Smaller effect size = A lot of error present
30
What do you do if the assumption of normality is violated?
Look if data is normally distributed or use the Shapiro-Wilk test.
Once found you can
- Remove outliers
- Transform data e.g. using a logarithmic transformation which is useful for reaction time data which often have a positive skew or use an Arcsine transformation which is useful for data in forms of proportions as it stretches out both tails of the distribution relative to the middle.
31
What do you do if the assumption of sphericity is violated?
When violated, there is a higher chance of making a Type II error therefore decreasing power.
Use the greenhouse-geisser or Huyhn-Feldt correction factor.
Test using Mauchly's test
Go ahead with ANOVA or use correction factor. If Epsilon is less than .75 then use Greenhouse Geisser correction line. If more than .75 then use Huyhn-Feldt.
32
When do you not go ahead with an ANOVA?
- Small sample size - Can't be confident that it is robust
- Data samples are non-normal in different ways e.g. positive and negative skew. Cannot correct this.
- When one variance is more than 4 times the other
- Unequal sample size in each condition - Can exaggerate heterogeneity of variance
Use non-parametric tests. Free of assumptions, but have less power. There is a not a non-parametric equivalent for mixed-design ANOVA.
33
What does a mixed factorial ANOVA do?
Compares several means when there are two or more independent variables
34
Why is unequal n a bad thing in an ANOVA?
It can exaggerate heterogeneity of variance
35
What kind of transformation would be most useful when normalising the distribution of RT data?
Logarithmic
36
If the Levene test was significant in a one-way between subjects ANOVA, what should you do?
Run a Kruskal Wallis test
37
What happens if you increase sample size?
Power increases
38
What is the difference between a main effect and an interaction effect?
Main Effect = Effects of each IV on the DV. Average across the IV then see if there is a difference.
Interaction Effect = The effect of the combination of the 2 IV's on the DV
39
What are simple effects?
Compare the first two points. If these differ significantly, can say that they are different.
40
When do you not test simple effects?
- When you have no reason to
- When you can tell the story by only talking about main effects
- No evidence of an interaction
41
What is homogeneity of covariance?
Multivariate version of the univariate assumption of homogeneity of variance.
42
What is the difference between randomised and matched groups design?
Randomised Groups Design
- Each individual has equal chance of being assigned to one group or the other
- Any random extraneous variables are assumed to be equally distributed across the groups and across experimental conditions.
Matched Groups Design
- Individuals in one group are matched with individuals in the other group on relevant variables
43
What is the purpose of ANCOVA?
Compare group means, but adjust those means for another variable that you expect to affect the outcome.
ANOVA looks for differences in group means, but ANCOVA looks for differences in adjusted means (adjusted for the covariate). Allows you to statistically control for a third variable which you believe will affect results.
Two main reasons to include covariates in ANOVA:
- Remove error variance due to extraneous variables. Can enhance the power and make it easier to detect the effect of the independent variable.
- Elimination of confounds - If any variables are known to influence the outcome variable, including them as covariates can remove these as explanations for the effect of interest
44
When do you use ANCOVA?
Only for between-participants
45
What is a one-way between participants ANOVA used for?
Examine differences between two or more independent groups.
Do the scores between people in different groups vary more than the scores of people within each group.
46
What is variance called in an ANOVA?
Mean square.
Divide the sum of squares by the df.
df = number of groups - 1
47
What is sum of squares?
Provides a sense of the amount of variation between groups and within groups
48
What is an F ratio?
Ratio of how good the model is to how bad it is (how much error there is)
Calculated by dividing the model mean squares by the residual means square (error)
49
What should you do if the assumption of homogeneity of variance is violated?
Find this out from Levene's test. If greater than 0.05 then variances are similar.
ANOVA is robust, so can usually go ahead. Make sure to report.
50
What is the difference between an independent and dependent variable?
IV = The variable being changed to controlled for by the experimenter
DV = Variable which changes in response to the IV. The one being tested
51
Describe what a two-way between ANOVA is
Use it to examine the differences between two or more independent groups on two IV's.
2 IV's and 1 DV
52
What does a 3x4 ANOVA mean?
IV has 3 levels
Other IV has 4 levels.
1 number for each IV. There are 2 IV's in a two-way ANOVA
53
What do you do to break down interactions?
Carry out a t-test or an ANOVA as they both find out how big the difference is compared to how much difference you would expect to find simply by chance.
54
How do you report interactions?
- State the interaction of each variable
- State interaction
- State the simple effects - 'A simple effect analysis showed that...'
55
What is a 2x2x2 design and how would you analyse this?
- 3 variables each has 2 levels
| - Analyse using a 3-way ANOVA
56
What do you do if there is a significant interaction?
Follow up with a simple effects analysis to see if true.
57
What are the assumptions of an ANCOVA?
Same as ANOVA, but add one.
- Independence of the covariate and treatment effect: Covariate and experimental variable must be independent from each other and not share the same variance. Test by carrying out ANOVA, but put covariate into the dependent variable box and keep one as a fixed factor. If ANOVA is not significant then the assumption is met.
- Important to randomly assign conditions to participants or match experimental groups on possible covariates.
58
Explain the differences between qualitative and quantitative research
Quantitative
- Usually has larger samples, more reliable
- Already have theory when conducting
- Focused on generalisation
- Belief that researcher is objective
- Deductive
Qualitative
- More valid as explore what something means for different people in different contexts.
- Inductive - Need research q, but has more explorative power.
- Can't make huge generalisation as need to link it to theory
- Purpose: Forumulate theory based on findings
59
How do you design a quantitative study?
- Develop research question, no hypotheses
- Sampling: Smaller samples used.
- Ethical considerations - Lack of anonymity as often meet in person. Need to ensure trust, no pressure.
- Raw data must be held securely, able to be removed
- Withdrawing data: Small samples means it has a greater impact
- Observation: People have reasonable expectation of privacy need to consider this.
60
What is the snowballing sampling method?
Interviewer asks participant if know anyone else who could take part.
Way to gain large numb of participants
Need to be aware of the groups
61
What are qualitative ways to collect data?
Observation
- Naturalistic: Unaware of observation
- Lab based: Controlled, aware of observation
- Field notes
- Inter-coder reliability
Interviews
- Online, face-to-face
- Preparing important
- Role of interviewer important
Focus Groups
- Preparation similar to interviews
- Role of researcher different
Naturally occuring data
- Speeches, social media, tv
- Recorded in some way, but researcher has no influence over.
- Participants natural behaviour in certain circumstances
62
Describe what Cohen's Kappa Coefficient is
Used to measure inter-rater reliability. Inter-rater reliability happens when data raters give the same score to the same items.
Assumes that raters are deliberately chosen.
Compensates for agreements made by chance whereas percent agreement does not.
Good agreement between two raters happens when the value is higher than 0.7
63
What steps are involved in content analysis?
- Developing a research question
- Developing coding frame
- Assessing reliability
64
Explain the positivism approach to interviews
- Real thoughts and cognitive processes. Talk is a route to cognition.
- Works with highly standardised questions and structured interview schedules
- Believes that talk provides reliable data on thoughts
- Not interested in interviewer's contribution (need to be neural, can come across as distant), only interviewee
- Problem is that there can be a variability in response
65
Explain the emotionalism approach to interviews
- Focused on expression of real feelings. Presentation of self.
- Assumes that the way they talk is the way they feel
- Deep unstructured interview
- Not interested in interviewer's contribution, only transcribe interviewees answers, but interviewer important for trust.
66
Explain the construtionalism approach to interviews
- Based on cognitive psychology
- Interested in how people construct versions of the world, people do things with words,
- Do not work with assumptions about the interviewees real thoughts or feelings
- Interested in construction and analyse them.
- People construct versions of reality with language which shapes our thoughts.
- Less concerned with structured format
Implications
- Read language as functional - sees things that are said as solutions to a problem
- Analysts task is to identify which interactional business being attended to.
- See interview as joint product
67
Describe the process of transcribing
Process of creating a written account of an audio recording.
Jefferson Transcription
- Used to indicate more subtle aspects of speech e.g. laughter, pauses
- Speech patterns
- Uses symbols
- Can be used in discourse analysis
- Can anomise details provided by interviewee
- Words in capital letters show very strong/loud emphasis
- Underlining = Emphasis of a word
- (.) = pause
68
What are the approaches to rigour?
Credibility
- Link findings with reality to demonstrate truth
Dependability
- Findings consistent with raw data collected
- Make sure if researchers carried out, would arrive at same findings
Confirmability
- Degree to which results could be confirmed or corroborated by others
Transferability
- How much the findings could be applicable to other contexts, situations, populations
69
Describe the process of triangulation in ensuring rigour
- Process of using multiple approaches to the gathering of data in a single study to improve credibility of conclusions.
4 different types
- Methods Triangulation: Check consistency of findings by different methods
- Triangulation of sources: Consistency examined within same method
- Analyst triangulation: Using multiple analyst's to review findings
- Theory/perspective triangulation: Using multiple theoretical perspectives to examine and interpret
70
Describe the process of member checking in ensuring rigour
When researchers return to the people who participated and run interpretations of data with them. Technique to increase validity, showing they are accurate and honest.
Advantages
- Corrects errors
- Provide additional information
- Preliminary findings can be summarised
Disadvantages
- Differing agendas can exist
- Good person vs good scholar
- Diff participants may have diff views
71
Describe the process of reflexivity in ensuring rigour
Involvement of researcher in data.
Encourage this by using multiple researchers, reporting in journal articles,
72
Explain the differences between content analysis and thematic analysis
Content
- Establishing categories + counting number of instances of each code
- Descriptive rather than analytical
- Superficial understanding of data
- Similar to quantitative ways
Thematic Analysis
- Involves looking for patterns across a dataset. Broad themes
- Analytical rather than descriptive
- More depth
- Data not dealt with in quantitative way
73
How do you carry out a mixed ANOVA in SPSS?
- Analyze > GLM > Repeated measures
- Define factors for within factor
- Put between-subjects factor in box
- Test assumptions for both mauchly's and levene's
74
How do you report the results of a two-way mixed ANOVA?
- Describe the experiment and the IV, DV. Give means and either give them in text or produce a table.
- State the statistical test e.g. 'A 2x2 mixed ANOVA was conducted on the scores'
State the main effects of the factors.
State the interaction
Look at interaction graph
Explain why these results occurred.
75
Explain how to carry out an ANCOVA within SPSS
- Analyse > GLM > Univariate
- Select usual options
- After done, include one variable as a covariate.
- Under options, display means and tick 'compare main effects' and select LSD to get pairwise comparisons.
76
Explain how you check the assumptions of an ANCOVA
Run a t-test or ANOVA with the covariate as the DV.
If ANOVA is not significant then the assumption is met as means they are dependent from each other.
77
How do you conduct a one-way between-participants ANOVA?
- Need to explore data to check parametric assumptions. Histogram and normality plots with tests.
- Analyse > GLM > Univariate
- Post-hoc Tukey
78
How do you report the results of a one-way between participants ANOVA?
- Explain the experiment and what statistical test used.
- Explain the variables
- Report any violations of assumptions if there are any.
- Go ahead with ANOVA bc robust to minor violations of assumptions.
- Provide the means if there are 3 or less.
- Then report ANOVA result.
- Post-hoc Tukey HSD test
79
How do you report violations of normality assumption?
"The Shapiro-Wilk test showed that the data in the ___ condition do not satisfy the assumption of normality, W(330) = .91, p < .001.
Similar for Levene's
80
Describe content analysis
- Quantitative way of analysing qualitative data involving establishing categories and frequency of instances in which they occur
- Coding allows to categorise occurences of a particular theme
- Changing data type makes it more objective, however can be seen as reductionist. Complex and detailed qualitative data gets reduced to numerical figures.
- Useful when working with large data set.
- Inter-coder reliability can be assessed using Cohen's Kappa coefficient
81
Describe thematic analysis
- Taking a body of text to analyse an existing theory.
- Involves organising the qualitative data into specific themes pre-identified by existing theory. Theories exist before analysis.
- Summarise data into distinct categories
- Bruan & Clarke say there are 6 steps
- Become familiar with data
- Code data
- Look for themes
- Review themes
- Finalise themes
- Report themes
82
Describe grounded theory
- Researchers intend to generate a theory based on data that gathered and analysed. No research q first.
- Develop theory about overall data which applies to it.
- Data collection and analysis happen at same time.
- Data-driven
