4. Experimental designs and control Flashcards Preview

PYB210 > 4. Experimental designs and control > Flashcards

Flashcards in 4. Experimental designs and control Deck (101):

what is an experiment?

An experiment is a strictly controlled study in which the ultimate aim is to infer causality on the part of the IV on the DV.
In other words, in order to say that changes in our IV CAUSED changes in our DV we need to make sure that any and (hopefully) all (if not most) alternative explanations have been accounted for


what is something we need to ensure that our study infers causality?

that it is internally and externally valid


what are the two main issues of validity?

internal validity and external validity


internal validity

how sound is the design, how strongly can we assert that changes in our DV are down to our IV and not other things we haven’t controlled for (i.e. extraneous variables)


external validity

how generalisable are our findings (tied in with representativeness of sample), how representative of the real world (tied in with how artificial our study is)


what is the relationship between internal validity, external validity and the artificiality of the study and its generalizability?

The more stringently we try to control or ensure internal validity, the potentially more artificial our study becomes and hence less representative of reality and hence less generalisable… and hence less EXTERNALLY valid.


what are the four steps to internal validity?

- Sound operationalisation of our DV
- Strong experimental design logic
- Sound operationalisation of our IV(s)
- Consideration and use of appropriate remedies to control for extraneous variables


are some weak experimental designs?

One group posttest only
one group pretest-posttest
posttest-only non-equivalent groups


what are the two ways of manipulating the IV?

experimental manipulation and individual difference manipulation


experimental maniplation

Experimenter determines which level of the IV a participant is tested at;


what are the two types of experimental manipulation

event and instructional


individual difference manipulation

A characteristic of the participant determines the level of the IV at which they are tested


three main types of strong experimental design

repeated measures, between groups, factorial (Mixed factorial)


repeated measures (within groups)

each participant tested at each level of the IV


Between groups

each participant tested at only one level of the IV



when theres more than one IV. May have all repeated measures IV or all between groups IVs



more than one IV with at least one IV manipulated between groups and at least one within groups.


what does mixed-factorial designs allow?

Allows examination of interplay between two or more IVs and the splitting up of these effects into interactions and main effects


what are the strengths of factorial designs?

o more than one independent variable allows for more precise hypotheses
o control of extraneous variables by including as an independent variable
o ability to determine the interactive effect of two or more independent variables


what are the main effects of the factorial design?

o the influence of one independent variable on the dependent variable
o one main effect for each IV in a study


what is the interaction effect of the factorial design?

o the joint, combined, or “interactive” effect of two or more independent variables on the dependent variable


what are the weaknesses of factorial designs?

using more than two independent variables may be logistically cumberstone.
high-order interactions are difficult to interpret


how do you maximise the chances of getting a true picture of how the IV affects the DV?

maximise the impact on our DV that is related to the IV (increase between group/condition/level variation)

Minimise variation in our DV that is not related to IV (compress within group/condition variation)


how many levels must an IV have to allow comparison of performance?

at least two


what is the fundamental thing to consider hen operationalising the IV?

how do you ensure that you will measure the IV's impact on the DV to its maximum effect?


how do you ensure that you will measure the IV's impact on the DV to its maximum effect?

To this end you need to try to ensure you include as extreme or distinctly separate levels of your IV as possible


what are the two variables that need to be separated

DV and V


what is the variable that needs to be compressed?

extraneous variable


what are the two forms of extraneous variables?

noise creating


noise creating , extraneous variable

randomly impact the DV, not related to the IV, but potentially create extra variation in the DV not due to the IV, want to minimise this


confounding - extraneous variable

impact the DV, related to the IV, potentially explaining changes in the DV that you would be expecting the IV to make, want to control for this by eliminating, keeping constant or building into study so can measure impact


what do extraneous variables lead to?

error - variations in measurement due to unwanted, uncontrolled or immeasureable factors need to be minimised


what variables and between groups and repeated measures design bring?

a unique set of extraneous variabes


what is the trouble with between groups?

Two separate groups of people could differ on a whole range of things. Both relevant and irrelevant to the study at hand


how do you make the groups in a between groups experiment as similar as possible?

through careful selecting of participants to the levels of IV


what are the processes of assigning individuals to experimental groups/conditions/levels of the IV?

self assignment, experimenter assignment, arbitrary assignment, random assignment/allocation


self assignment

subjects select treatment group


experimenter assignment

experimenter selects with treatment group


arbitrary assignment

selection based on seemingly non-relevant criteria


random assignment/allocation

ensuring that every member has an equal chance of being assigned to any group.


what issue does self-assignment, experimenter-assignment and arbitrary assignment carry?

bias to confound results


what is the benefits of random assignment?

provides maximum insurance that groups are equal
eliminates systematic differences between groups
randomly distributes extraneous variables across groups
reduces bias


what is matching?

using any of a variety of techniques to equate participants in the treatment groups on specific variables. This should be done with variables thought to be related to the IV or may confound the IV


what are the advantages of matching?

controls for the variables on which participants are matched
increases the sensitivity of the experiment


what are the advantages of individual/perception mathching?

groups equated on potential extraneous variables


what are the disadvantages of matching?

• identifying the variables on which to match
• difficulty matching participants increases as the number of variables on which to match increases
• in generalizability of results?


what is blocking?

building the extraneous variable into the research design. Make the extraneous variable another IV in the study. This should only be used when you are interested in the effect of the extraneous variable


benefits of a repeated measure design

- It eliminates the problem of group differences arising from individual differences in group make-up
- Requires fewer participants to have good statistical power (because it is a more refined measure of the impact of IVs with a major source of error removed)
- Having the same participants take part in each condition or level of your IV eliminates the problems of individual differences and needing to deal with these via random allocation or matching


problems with repeated measure designs

participants may perform differently in each condition based on their prior experiences in the study
can lead to sequencing effects


what are sequencing effects?

order effects (practice and fatigue) and carry over effects


practice effects

Where completing measures of the same DV or outcome variable multiple times may lead to the participant becoming practiced at the measure.
If the DV is performance based e.g. reaction time then differences improvements may occur simply due to the practice rather than being due to different conditions of the IV


fatigue effects

where repeated completion of the IV measure of task may lead to boredom or tiredness


what is the solution to the problems of sequencing effects in repeated measures designs?




•In order to control for sequencing effects we can use counterbalancing
•breaking our sample into subsets who will experience the different conditions in different orders
•By then collapsing all the data together again hopefully we can also test for the existence of sequencing effect as well as collapsing across them


randomized counterbalancing

sequence of conditions is randomly determined for each participant


what are different types of counterbalancing?

randomised, intrasubject, complete, incomplete


intrasubject counterbalancing

participants take treatments in ore than one order. May not be feasible with longer treatment sequences


complete counterbalancing

all possible sequences of treatment are used. Participants randomly assigned to the sequence
N! = N multiplied by each number below it
rarely used with more than 3 conditions because number of possible sequences (N!) too large


incomplete counterbalancing

most common technique
Not all possible sequences are used.
requires a criteria and formula to determine the sequence


what is the criteria for incomplete counterbalancing?

• each treatment condition must appear an equal number of times in each ordinal position and
• each treatment condition must precede and be followed by every other condition an equal number of times


what are the two carry over effects of counterbalancing?

simple and differential


what is the simple carry over effect for counterbalancing?

Where performance on the DV in one condition is contaminated by the effects of the previous condition


What is the differential carry over effect for counterbalaning?

Where the carry-over effects of one condition of the IV differ depending on the order in which the conditions are completed.


why is counterbalancing a great tool?

It allows you to control for order effects by providing data from a range of different orderings of measurement


what happens when time is an IV

participants provide a baseline measurement and then complete measurements at numerous time points during and after the treatment program.


what is the problem with time as an IV?

cant counterbalance time


what are issues when having time as an IV?

maturation, history, statistical regression, mortality



• Changes due to natural development, natural expected improvement over time etc
• If participants improved over time the question becomes whether or not it is as a function of the IV, or just a natural improvement that would have occurred anyway


history (external events)

• External events that affect participants during the study
• Socio-historical-economic changes relevant to outcome/DV
• While participants were taking part in a program to reduce smoking there was a massive price rise in cigarettes


statistical regression

• Refers to tendency to move up or down towards the mean over time
• In other words someone who was scoring below or above par on a measure is potentially likely to move towards the mean of that variable over repeated measurements



• Attrition from the study: Not all participants who take part in the first measurement points will remain in the study until the bitter end


what are other effects to an experiments validity?

• Experimenter Effects
• Participant Effects
• Situational Effects


experimenter effect

• The experimenter as part of the experiment
• Measurement, attributes and expectancies


measurement issues regarding the experimenter effect

o problem with equipment or errors in manual recording of data between measurement point or between participants on the part of the experimenter


how does one control measurement issues regarding the experimenter effect?

• make researchers aware of making careful observations (training) multiple data records (video camera, computers) or observers have participants make responses on computer


attributes effect with regard to experimenter effect

participants responding differentially to different experiments within the study


how does one control attribute errors?

use the same experimenter in all treatment conditions (unless the treatment condition interacts with attributes.


what are expectancies with regard to the experimenter effect?

are the likely response of participants to the study and its manipulations may lead to subtle differences in the way the experimenter interacts with participants which lead to differences in outcomes


what are the two sub categories of expectancies with regard to the experimenter effect?

rosenthal effect and the golem effect


rosenthal effect

differential attitude or attention conveyed to participant expected to respond most favourable to the study


golem effect

differential attitude or attention conveyed to participant expected to respond least favourable to the study


what are controls of the experimenter expectancies?

double-blind method, partial blind technique, automation


double-blind method

neither the experimenter nor the research participant is aware of the treatment condition administered to the participant


partial blind technique

a method whereby knowledge of each research participant’s treatment condition is kept from the experimenter through as many stages of the experiment as possible



the technique of totally automating the experimental procedures, so that no experimenter-participant interaction is required


what are problems that come with participant effects?

demand characteristcs, social desirability, hawthorne effect


demand characteristics

participant gets an inkling about what the study is aiming to achieve and performs in a way to conform to those expectations


social desirability

participant performs in a way they think will be most pleasing to experimenter, or will paint them in the best light


hawthorne effect

participant improves or changes performance on the outcome purely as a function of the attention received for being in the study and not as a function of the nature of the manipulations of the IV


what are the controls of the participant effect?

single-blind double-blind, deception


single blind study

The participant is not made aware of the true purpose of the study or the nature of the group in which they are in


double-blind method

neither the experimenter nor the research participant is aware of the treatment condition administered to the participant



o omission of or altering the truth of information given to the participant during a research study
o used when there is no other way to gain the knowledge and risk does not outweigh the benefit of the information
o must keep the false information constant for all participants


situational effects

impacts o environmental and timing differences on participants' outcome scores.
Includes time of day, weather, lighting, noises


how are situational effects controlled?

keep these constant for all participants or counterbalancing


what does control groups allow?

allows you to directly examine the extent to which changes would have occurred without your intervention/s as well as potentially examining a whole raft of other extraneous variables


what is a crucial thing to ensure when forming a control group

that it is equivalent


if you are to have multiple control groups, what are the conditions of the two groups?

no intervention
placebo intervention


what does having multiple control groups counter for?

the hawthorne effect


what are the key features of whtin groups designs?

o Minimise individual difference error
o Problems with order effects, practice, cumulative effects, carry-over effects, history, maturation, attrition/mortality
o Can control these things somewhat with counter-balancing, random order variation


what are the key features of a between groups design?

o Eliminate problems of WG
o Individual differences come into play
o Ensuring comparability of groups/matching