PSYC3007 - Research Methods 3 Flashcards
1
Q
Why do formal research in psychology?
A
Because it enables use to test and justify our claims and to discard propositions that are false. Research is what distinguishes psychology from other bases for claims about the causes of behaviour, and the effectiveness of treatment.
2
Q
Why is it important that the details of a study’s methods, analysis and results are published?
A
Our conclusions can be scrutinised by others. We can determine the justifiability of other’s conclusions. It allows us to judge the strength of effects It makes us and other psychologists accountable.
3
Q
Why is it important to learn about research methods even if we don’t intend to do research?
A
It allows us to be discerning consumers of the products of others research.
4
Q
The truthfullness of the results and conclusions that derive from a piece of research depends on… ?
A
The quality of the design The quality of the measures The appropriateness of the statistical analyses The nature (e.g., size) of effects The match between the results and the conclusion
5
Q
What are the three essential conditions for establishing causation?
A
- Covariation: two events must occur/change together (including non-occurrence) 2. Temporal order: the cause must precede the effect 3. Elimination of plausible alternative explanations for covariation
6
Q
If the only substantial difference between 2 conditions is A (an independent variable) does that allow us to determine that A is the cause of B (the dependent variable)
A
Yes.
7
Q
What is Internal Validity?
A
The extent to which we can say that (any) effects on the DV were caused by the IV (as opposed to something else)
8
Q
What is selection threat? How can it be controlled?
A
Anything that can result in non-equivalent groups in different conditions (it reflects the method of condition assignment) (A threat to internal validity) It can be controlled with Random Assignment.
9
Q
What is History Threat? How can it be controlled?
A
Another event could coincide with a study and affect its results. (A threat to internal validity) It can be controlled with random assignment
10
Q
What is Maturation? How can it be controlled?
A
Refers to changes in participants as a result of the passage of time. (A threat to internal validity) It can be controlled with randomisation and a control group.
11
Q
What is regression towards the mean? How can it be controlled?
A
Refers to the fact that scores that are very different from the mean on a first measurement occasion can be nearer on the mean when the measure is taken again.
12
Q
What is selection threat? How can it be controlled?
A
Selection is anything that can result in non-equivilant groups in different conditions. Any non-random method of allocating participants to conditions leads to selection threat. E.g., testing YOGA program on workers in a large company whereby volunteers go to the yoga-condition and non-volunteers go to the control group. IT can be controlled by random allocation (of sufficient numbers), because this should produce equivalent groups. Note: DON’T CONFUSE this with ways of ‘selecting’ people from a population from a study
13
Q
What is a confound?
A
- Another variable that is RELATED to the IV (e.g., differs between conditions).
- Not just any extraneous/uncontrolled variable that could apply randomly to different individuals regardless of condition. (e.g., NOT, 1 person breaks a leg, 1 person gets promotion)
e.g., rats have tail removed die and rats who don’t have their tail remove survive……do rats need their tail to survive? NO, surgery and human handling are confounds.
14
Q
What is instrumentation threat?
A
Changes in the way a measure is applied or intepreted.
e.g., an experimenter gets better at encoding a behaviour, or a patient gets less diligent over time recording in a diary
15
Q
What are testing effects?
A
Changes that occur in a measure/score over 2+ testing occassions, BECAUSE of the earlier measure.
16
Q
What are demand effects?
A
Responding in a way that is consistent with the researcher’s expectations as a result of participants knowing/guessing what the researcher expects.
17
Q
What are non-specific treatment effects?
A
Benefits of warmth, attention, support, reassurance and the mechanics of homework exercises (rather than from the specific treatment itself)
18
Q
What are placebo effects?
A
Changes owing to the receipt of a treatment (not owing to the actual treatment). People can feel better just because they know they’re receiving a treatment. Placebo effects result from participants’ own expectations about what will happen.
19
Q
What is attrition?
A
(aka ‘mortality’) drop outs (people who leave the study before all of the intended data are collected from them). Some attrition is common in studies that use repeated measures over time. It’s not necessarily a problem: it depends on the pattern and possible causes.
20
Q
What is experimenter bias?
A
Researchers might inadvertently administer experimental instructions differently in different conditions, or administer or interpret measures differently in different conditions (this only applie when discretion/judgment is used in determining scores)
21
Q
How do we (in general) control for threats to internal validity?
A
- Ensure that threats apply similarly in all conditions.
- The only difference should be on the IV.
- A control condition should be used if appropriate - placebo should be similar to actual treatment
- Counterbalencing (when experimental conditions are varied within subjects, the order should be varied when feasible.
- Blind assessment prevents administration or intepretation bias
- Blind participants
- Rigorous training for measures requiring skilled delivery or intepretation.
22
Q
What are the features of an appropriate placebo?
A
Is similar to the real treatment in:
- Creadibility
- Expectations aroused
- TIme commitment/effort
- Attention, warmth etc from the researcher
- Ideally, method of administration (drug/individual)
23
Q
Do all treatment studies need a placebo?
A
No
Depends on research question
- relative effects of A and B (comparison study)
- is A better than placebo
- Correlation study - control condition mostly irrelevant
24
Q
How do we control for researcher/bias (Where applicable)?
A
- Standardisation instructions to participant
- Blind assessment/scoring
Very good training in the use of measures requiring skill (to avoid bias, instrumentation effects)
25
what does it mean to say a threat is controlled?
That the threat applies similarly to differet conditions, or more or less randomly across different people in a study, it has been controlled.
26
What is a quasi-experimental design? What is it's major flaw?
**Ps not randomly allocated to levels of the IVs**
* Sometimes because the IV can't be manipulated E.g emplyed vs. unemployed, asthma vs. not
**Ps might differ in ways other than the IV**
27
What is a Factorial Design?
2 or more CATEGORICAL (factors) IVs are manipulated (or measured) AND all combnations of the IVs are tested.
"an experiment whose design consists of two or more factors, each with discrete possible values or "levels", and whose experimental units take on all possible combinations of these levels across all such factors."
28
How are factorial designs written?
By specifying the no. of levels of each IV
e.g., a 2x2 design has 2 IVs and each has 2 LVLs
29
How do you determine the number of cells in a factorial design?
Multiply the number of levels of each IV together
e.g.
* 2x2 has 4 cells
* 2x3 has 6 cells
* 2x2x2 has 8 cells i.e., (2x2)x2
* 3x3 has 9 cells
30
What is the differece between 'between-subjects', 'within-subjects' and 'mixed' manipulation of an IV?
Between-subjects = Each Participant is only in one level of the IV
Within-subjects = Each participant participates in each level of the IV
Mixed = At least one IV is varied between subjects and at least one is varied within subjects (e.g., pre-post time)
31
What are blind designs?
When participants/experimenters/assessors are not aware of condition.
Single-blind = either participants or assesor/experimentor does not know (not both)
Double-blind = both participant and assessor/experimentor do not know
32
What is a cover story?
Not telling participants what the hypothesis/aims of the study are
33
What is counterbalencing?
Equal representations of the possible order of presentaton of conditions in within Ps design.
34
What are manipulation checks?
* If trying to manipulate the mood, then measure the mood.
* If using placebo condition to control for expectations, measure expectations. (expectations, optimism, credible)
35
What are matched groups (i.e., matched pairs)?
* Participants in different conditions are matched on potential relevant extraneous variables (e.g., education, age, drug use)
* Note: this is rarely needed and can be difficult to intepret
36
What is construct Validity?
* Extent to which measures and manipulation actually reflect the theoretical constructs of interest (both IVs and DVs) .... the accuracy/truthfullness of the measure.
37
In writing a design statement, what do you need to indicate?
* The IVs: give each a name
* The number of levels of the IV(s), and their identity
* Whether each IV was varied between or within subjects
* The DV(s)
* Experiment or not (more so what is randomised, what is not, unless obvious)
38
What are the different types of internal validity?
* Face validity
* Content validity
* Criterion validity (concurrent and predictive)
* Convergent validity
* Discriminant (or divergent validity)
39
What is the "operationalisation" of 1. measurement and 2. experimental manipulation?
1. The precise activities or operations used to measure a construct (e.g., exactly how 'anxiety' or 'happiness' or 'anti-social behaviour' or 'task effort' was defined and measured)
2. The precise activities or operations used to manipulate an independent variable in an experiment (e.g., exactly how 'positive' versus 'negative' feedback to participants was manipulated)
40
What is face validity?
Whether the measure *appear* to be related to construct of interest.
41
What is Content validity?
The extent to which a measures assesses the range of characteristics thought to represent the construct of interest.
e.g., If we attempted to measure 'honesty' simply by asking the question "have you ever told a lie"? our measure would not have content validity, because there are many forms of dishonesty other than telling a lie.
(in addition, the measure wouldn't have face validity: all of us have told lies at some time, even if it was just to preserve someone's feelings or solely as a child)
42
What is criterion validity? describe both types?
The relationship between the measure and an external, separate criterion that presumably reflects or depends on the construct.
**Concurrent** - the correspondence between the measure and a currently available criterion. For example, a measure of honesty, if valid, should differ between convited embezzlers and a group of people that is presumed to be honest (e.g., nuns)
**Predictive** - The correspondence between the measure and a future criterion, e.g., its ability to predict behaviours in advance/ (this is possibly the strongest evidence of validity)
43
What is convergent validity?
The extent to which the measures correlates with other variables (usually simply other measures) that are predicted to be related to the construct. It's the degree of overlap with other measures of the same construct.
44
What is discriminant (or divergent) validity?
The extent to which the measure discriminates between the target construct and other, different constructs. It shouldn't correlate highly with measures of different constructs. For example, a measure of self-esteem should not correlate (or only correlate weakly) with anxiety, social class, IQ, or the age at which someone learned to ride a bicycle.
45
What is external validity?
The extent to which our results can be regeralised to the relevant population of interent or to other relevant settings or times. This is affected (among other things) by:
* Representativeness of the sample
* Whether influences operate in the testing or measurement situation that do not apply in other settings (or vice versa)
46
What is ecological validity?
It is related to external validity, although the two are no identical. It is the extent to which the results can be generalised to real life settings.
47
What factors is reliability affected by?
* Whether questions designed to measure the same construct really do
* Random measurement error (e.g., mood, misunderstading a question, distraction)
* The number of items (larger numbers tend to increase reliability estimates)
* Clarity of instructions and questions, and standardisation of instructions.
48
Does reliability guarantee validity?
NO! can just be consistently (reliably) wrong.
49
What is the problem with this questionaire question?:
1. How close is your relationship with your mother?
very close, quite close, fairly close, not very close, very distant
Unwarranted assumption - some people may not have a mother.
50
What do descriptive statistics do? and what DON'T they do?
Descriptive statistics allow us to describe and summarise scores on a variable, or the relationship between variables. They are used because they make a set of data easier to understand.
e.g., in the survey, 38% of respondents said that they approved of the job being done by the Prime Minister, 42% said that they disapproved, and 20% were undecided.
They DON'T tell us whether a particular difference or linear relationship could have arisen by chance (Rather than reflecting a 'real' effect)
51
What do Inferential statistics do? and what DON'T they do?
Inferential statistics simply tell us whether an effect (or relationship) reflects a 'real' effect, or could simply have arisen by chance owing to normal variation in a sample or population. Related to this, they indicate whether we can infer that an effect observed in a sample is also likely to occur in the population of interest.
(there will be some difference between the groups on almost any measure, simply owing to chance)
They DON'T...... t and f values and statistical significance levels (p) do not indicate the **size, direction or importance** of an effect....this is what descriptive statistics and our judgement does!
52
In statistics, what is significance?
The significance level or p is the probability that an observed effect arose by chance if there really is no effect.
for example, the p from a t-test is the likelihood that an observed difference between means arose by chance if there really is no difference, If there is little likelihood (less than or equal to 5%: p equal or less that .05), we consider it to be a 'real' (significant) effect. Incidentally, the criterion level of p, p equal or less than .05, is called the *alpha* level.
53
What are the three (main) types of descriptive statistics?
* **Mean (M), Median & mode** - typical score (on a continuous measure).
* **Percentage or proportions** - of cases that fall into particular categories or frequencies
* **Correlation coefficients** (r) and **regression coefficients** (B and Beta) - these describe the nature of a linear relationship between continuous variables. Correlations indicate the direction and strength of the relationship (e.g., .2 is small, .35 is medium, .5+ is large) Regressions coefficients also indicate direction. Bs tell us about the effect in the original scales of measurement. Betas can indicate the relative strength of different predictors. More about these later.
54
What are indices of variability?
Values that indicate the degree of variation in scores. The standard deviation (SD) is the most interpretable of the measures of variability. When reporting means, give their SDs too.
55
What is 'variance'? (as an indice of variability)
Variance is roughly the 'average' of all squared deviations from the mean. It's calculated by squaring the difference between each score and the mean, adding the squared differences (squared deviations) together, and dividing by n-1. Statistical procedures use the variance, and the SD is calculated from it. However, it's less interpretable or meaningful than SD.
56
What is percentage (or portion) of variance explained?
The percentage of variance in the DV that can be explained by an IV.
57
How do you find the percentage or proportion of variance explained for CORRELATIONS?
For correlations (which handle only 2 variables at a time), the proportion of variance explained is simply r2 (i.e., the square of the correlation coefficient).
for example, if the correlation between body weight and dietary intake in calories per day is r=.66, then calorie intake explains (.66x.66=) .4356 (as a proportion) or 43.56% of teh variance in body weight. In other words, calorie intake explains a substantial proprtion of the variance, BUT 56.44% of the variance in weight is left unexplained, and so weight must also be affected by some other variable or variables ( such as genetic predisposition to store fat and to be a particular weight, and exercise)
58
How do you find the percentage or proportion of variance explained for REGRESSION?
The proportion of variance explained is given in the statistics R2 and/or R2change, which statstical packages such as SPSS report for us.
59
How do you find the percentage or proportion of variance explained for ANOVA?
There are a number of indices of variance explained. One is eta2 (n2) or the proportion of total variance in the DC that's explained by an IV.
For factorial ANOVAs, SPSS reports partial n2, which is the proportion of variance in the DV that is explained by a particular IV which ignoring the effects of any other IVs in the analysis.
60
What two assumptions do Parametric tests make? what happens if these assumptions are not mett?
1. Scores on continuous variables (that are used in the analysis) are normally distributed
2. The variability (or variance) in scores is similar for different conditions (Levene's test)
If these assumptions are violated (not true), then the outcome of a parametric test can be misleading (wrong). If the data do deviate markedly from these assumptions, we could use a nonparametric test (there are other potential solutions to).
61
What are the two broad classes of parametric tests?
1. Tests that assess the significance of the **difference between means** (t-tests and ANOVAs)
2. Test that assess the significance of the **linear relationship between continuous variables** (correlations and regressions)
62
What is the procedure of SD?
1. Subtract mean from each person score = difference score
2. square difference scores for each person (gets rids of negatives)
3. Add them
4. Divide by N-1 to get the rough average of squared devaitions = variance
5. Squared variance to get SD
63
When you have a Categorical DV and a Categorical IV what is the appropriate statistical technique and statistical aim?
Chi-square
Compare proportions betwen categories.
64
If you have a continuous DV a single categorical IV with only 2 levels, what is the appropriate statistical test? what is the statistical aim?
t-test (or oneway ANOVA)
Test difference between means for the 2 levels
65
If you have a continunuous DV and a single categorical IV with 3 or more levels, what statistical technique would you use? what is the statistical aim?
Oneway ANOVA
Test whether there are differences somewhere among level means.
66
If you have a continuous DV and two or more categorical IVs, what statistical technique do you use? and what is the statistical aim?
Factorial ANOVA
Test main effects & interaction
67
If you have continuous DV/s and continuous IV/s what statistical technique should you use? what is the statistical aim?
Regression or correlation
Assess linear relationship
68
How can you test the assumption of normality (i.e., a bell-curve/normal distribution)?
* Plot the scores in a histogram (using SPSS) and use visual inspection.
* Also, there are statistic that can be requested that quantify the deviation from normality ....but we won't cover these.
69
What is the difference between positive and negative skewness (of a graph)? what does it look like and what does it mean?
* **Negative skew**: The left tail is longer; the mass of the distribution is concentrated on the right of the figure. It has relatively few low values. The distribution is said to be left-skewed, left-tailed, or skewed to the left.
* **Positive skew:** The right tail is longer; the mass of the distribution is concentrated on the left of the figure. It has relatively few high values. The distribution is said to be right-skewed, right-tailed, or skewed to the right.
70
To meet the assumption of 'homogeneity of variance' should the Levene's test (and statistical tests of departure) be significant or non-significant?
NON-SIGNIFICANT
71
Under what circumstances do **hetereo**geneity of variance (i.e.., difference between conditions in the variance in scores on the DC) and non-normal distrubutions for DCs pose a greater threat to the 'validity' of the statistical outcome?
* When the number per condition is small
* when the numbers per condition are markedly different
* The direction of skewness varies between conditions
* Heterogeneity of variance and non-normality coexist
72
An ANOVA/t-test is used to assess the difference between \_\_\_\_
means
73
What does 'ANOVA' stand for?
Analysis of Variance - it compares the means (on continous DVs) of conditions (categorical IVs) and considers why they differ.
74
What parametric test would you use to assess the difference between means where the IV has only 2-levels and is varied between Ps
AND
the statistical aim is compare means between levels?
Independent groups t-test (ore oneway ANOVA)
75
What parametric test would you use to assess the difference between means where the IV has 3 + levels and is varied between Ps
## Footnote
AND
the statistical aim is compare means between levels?
One-way ANOVA
76
What parametric test would you use to assess the difference between means where the IV has only 2 levels and is varied within Ps
## Footnote
AND
the statistical aim is compare means between levels?
Paired Samples t-test
77
What parametric test would you use to assess the difference between means where the IV has only 3 + levels and is varied within Ps
## Footnote
AND
the statistical aim is compare means between levels?
Oneway repeated measures ANOVA
78
What parametric test would you use to assess the difference between means where there is more than one IV, each IV has 2+ levels and is varied between Ps
## Footnote
AND
the statistical aim is to test main effects and interactions?
Factorial ANOVA (e.g., 2-way, 3-way)
79
What parametric test would you use to assess the difference between means where there is more than one IV, each IV has 2+ levels and is varied within Ps
AND
the statistical aim is to test main effects and interactions?
Repeated measures ANOVA
80
What parametric test would you use to assess the difference between means where there is more than one IV, each IV has 2+ levels and is varied both within and between Ps
AND
## Footnote
the statistical aim is to test main effects and interactions?
Mixed ANOVA (split plot ANOVA)
81
What is the purpose of error bars?
provide information about variability
represents error in measurement (in estimation of population mean)
indicate whether means likely to be significantly different.
82
What do 95% confidence interavals show/mean/represent?
they define the range of values within which the true mean is likely to lie on 95% of occasions.
95% CI indicate
1. error of measurement (in estimation of population mean)
2. whether means are likely to differ significantly
83
if the confidence interval bars for two means overlap by more than 1/4 is the difference between means likely to be significant or non-significant?
Non-significant
84
what do describtive statistic do? what what don't they do?
Allow us to describe and summarise scores on a variable, or the relationship between variables. They are used because they make a set of data easier to understand.
They DON'T tell us whether a particular difference or linear relationshop could have arisen by chance (rather than reflecting a 'real' effect)
85
What do inferential statistics do, what don't they do?
they do tell us whether an effect or relaionship reflects a 'real' effect, or could simply have arisen by chance owing to normal variation in a sample population.
They don't give any descriptive stats.
86
What is significance?
The significance level or 'p' is the probability that an observed effect arose by chance if there really is no effect.
87
What are the three types of descriptive statistics and when are they used?
1. Mean (M), median & mode - tell us typical score on a continuious measure for comparing means between conditions.
2. Percentages or proportions of cases that fall into particular categories (or frequencies) - compare between catefories with different no. of cases.
3. Correlation coefficients (r) and regression coefficients (B and beta) - describe the nature of a linear relationship between continuous variables. direction and strength. Bs tell us about the effect in the original scales of the measurement. Betas can indicate the relative strength of different predictors.
88
parametric tests assume that.......?
1. scores on continuous variables (that are used in the analysis) are normally distibuted
2. the variability (or variance) in scores in similar for different conditions (there are tests available to assess this, including in SPSS
if these assumptions are violated, then the outcome of parametric test can be misleading. If the data do deviate markedly from these assumptions, we could use a nonparametric test.
89
What are the broad classes/types of parametric tests?
1. tests that assess the signifiance of the difference between means (t-test and ANOVAs)
2. Tests that assess the significance of the linear relationship between continuous variables (correlations and regressions)
90
What are the assumptions of an ANOVA/t-test?
1. Scores on the DV are (roughly) normally distributed) i.e., bell curve)
2. Variance or variability is (roughly) similar for the different conditions(the assumption of 'homogeneity of variance')
91
In order to meet the homogeneity of variance assumption (i.e. in ANOVA) the levene's test should be \_\_\_\_\_\_\_\_\_\_\_\_\_?
Non-significant
92
In what situations does heterogeneity of variance(different levels of variance between conditions) pose a greater threat to the validity of the statistical outcome?
* the n per condition is small
* the Ns per condition are markedly different
* the direction of skewness varies between conditions
* Heterogeneity of variance and non-normality coexist.
93
What is the appropriate parametric test to assess the difference between means where the IV is varied between subjects and there is one IV with 2 levels?
Independent groups t-test (or one-way ANOVA)
94
What is the appropriate parametric test to assess the difference between means where the IV is varied between subjects and there is one IV with 3 levels?
Oneway ANOVA
95
What is the appropriate parametric test to assess the difference between means where the IV is varied within subjects and there is one IV with 2 levels?
Paired samples t-test
96
What is the appropriate parametric test to assess the difference between means where the IV is varied within subjects and there is one IV with 3+ levels?
Oneway repeated measures ANOVA
97
What is the appropriate parametric test to assess the difference between means where the IV is varied between subjects and there are 2 IVs with 2+ levels on each?
Factorial ANOVA (2-way, 3-way etc)
98
What is the appropriate parametric test to assess the difference between means where the IV is varied within subjects and there are 2 IVs with 2+ levels on each?
Repeated measures ANOVA
99
What is the appropriate parametric test to assess the difference between means where the IV is varied both between and within subjects (i.e., mixed design) and there are 2 IVs with 2+ levels on each?
mixed ANOVA (split plot ANOVA)
100
on the ANOVA results table, the degrees of freedom (df) on the same line as the releveant F is the \_\_\_\_\_\_-groups df. Whilst the df given on the line labelled 'Error' is the \_\_\_\_\_\_-groups df.
Between-groups; within-groups
101
What is a moderator?
The moderator is the variable (another IV) that affects the primary IV-DV relationship in which we're interested. In other words, the moderatory affects the relationshop between the key IV and the DV. We say that two IVs interact. We could just as correctly say that the IV and the moderator interact.
(Moderation is closely related to interaction, however it's a more theorestical idea in that it reflects our focus in posing and answering a research question)
102
