4. Experimental designs and control Flashcards Preview

PYB210 > 4. Experimental designs and control > Flashcards

Flashcards in 4. Experimental designs and control Deck (101):
1

what is an experiment?

An experiment is a strictly controlled study in which the ultimate aim is to infer causality on the part of the IV on the DV.
In other words, in order to say that changes in our IV CAUSED changes in our DV we need to make sure that any and (hopefully) all (if not most) alternative explanations have been accounted for

2

what is something we need to ensure that our study infers causality?

that it is internally and externally valid

3

what are the two main issues of validity?

internal validity and external validity

4

internal validity

how sound is the design, how strongly can we assert that changes in our DV are down to our IV and not other things we haven’t controlled for (i.e. extraneous variables)

5

external validity

how generalisable are our findings (tied in with representativeness of sample), how representative of the real world (tied in with how artificial our study is)

6

what is the relationship between internal validity, external validity and the artificiality of the study and its generalizability?

The more stringently we try to control or ensure internal validity, the potentially more artificial our study becomes and hence less representative of reality and hence less generalisable… and hence less EXTERNALLY valid.

7

what are the four steps to internal validity?

- Sound operationalisation of our DV
- Strong experimental design logic
- Sound operationalisation of our IV(s)
- Consideration and use of appropriate remedies to control for extraneous variables

8

are some weak experimental designs?

One group posttest only
one group pretest-posttest
posttest-only non-equivalent groups

9

what are the two ways of manipulating the IV?

experimental manipulation and individual difference manipulation

10

experimental maniplation

Experimenter determines which level of the IV a participant is tested at;

11

what are the two types of experimental manipulation

event and instructional

12

individual difference manipulation

A characteristic of the participant determines the level of the IV at which they are tested

13

three main types of strong experimental design

repeated measures, between groups, factorial (Mixed factorial)

14

repeated measures (within groups)

each participant tested at each level of the IV

15

Between groups

each participant tested at only one level of the IV

16

factorial

when theres more than one IV. May have all repeated measures IV or all between groups IVs

17

Mixed-factorial

more than one IV with at least one IV manipulated between groups and at least one within groups.

18

what does mixed-factorial designs allow?

Allows examination of interplay between two or more IVs and the splitting up of these effects into interactions and main effects

19

what are the strengths of factorial designs?

o more than one independent variable allows for more precise hypotheses
o control of extraneous variables by including as an independent variable
o ability to determine the interactive effect of two or more independent variables

20

what are the main effects of the factorial design?

o the influence of one independent variable on the dependent variable
o one main effect for each IV in a study

21

what is the interaction effect of the factorial design?

o the joint, combined, or “interactive” effect of two or more independent variables on the dependent variable

22

what are the weaknesses of factorial designs?

using more than two independent variables may be logistically cumberstone.
high-order interactions are difficult to interpret

23

how do you maximise the chances of getting a true picture of how the IV affects the DV?

maximise the impact on our DV that is related to the IV (increase between group/condition/level variation)

Minimise variation in our DV that is not related to IV (compress within group/condition variation)

24

how many levels must an IV have to allow comparison of performance?

at least two

25

what is the fundamental thing to consider hen operationalising the IV?

how do you ensure that you will measure the IV's impact on the DV to its maximum effect?

26

how do you ensure that you will measure the IV's impact on the DV to its maximum effect?

To this end you need to try to ensure you include as extreme or distinctly separate levels of your IV as possible

27

what are the two variables that need to be separated

DV and V

28

what is the variable that needs to be compressed?

extraneous variable

29

what are the two forms of extraneous variables?

noise creating
confounding

30

noise creating , extraneous variable

randomly impact the DV, not related to the IV, but potentially create extra variation in the DV not due to the IV, want to minimise this

31

confounding - extraneous variable

impact the DV, related to the IV, potentially explaining changes in the DV that you would be expecting the IV to make, want to control for this by eliminating, keeping constant or building into study so can measure impact

32

what do extraneous variables lead to?

error - variations in measurement due to unwanted, uncontrolled or immeasureable factors need to be minimised

33

what variables and between groups and repeated measures design bring?

a unique set of extraneous variabes

34

what is the trouble with between groups?

Two separate groups of people could differ on a whole range of things. Both relevant and irrelevant to the study at hand

35

how do you make the groups in a between groups experiment as similar as possible?

through careful selecting of participants to the levels of IV

36

what are the processes of assigning individuals to experimental groups/conditions/levels of the IV?

self assignment, experimenter assignment, arbitrary assignment, random assignment/allocation

37

self assignment

subjects select treatment group

38

experimenter assignment

experimenter selects with treatment group

39

arbitrary assignment

selection based on seemingly non-relevant criteria

40

random assignment/allocation

ensuring that every member has an equal chance of being assigned to any group.

41

what issue does self-assignment, experimenter-assignment and arbitrary assignment carry?

bias to confound results

42

what is the benefits of random assignment?

provides maximum insurance that groups are equal
eliminates systematic differences between groups
randomly distributes extraneous variables across groups
reduces bias

43

what is matching?

using any of a variety of techniques to equate participants in the treatment groups on specific variables. This should be done with variables thought to be related to the IV or may confound the IV

44

what are the advantages of matching?

controls for the variables on which participants are matched
increases the sensitivity of the experiment

45

what are the advantages of individual/perception mathching?

groups equated on potential extraneous variables

46

what are the disadvantages of matching?

• identifying the variables on which to match
• difficulty matching participants increases as the number of variables on which to match increases
• in generalizability of results?

47

what is blocking?

building the extraneous variable into the research design. Make the extraneous variable another IV in the study. This should only be used when you are interested in the effect of the extraneous variable

48

benefits of a repeated measure design

- It eliminates the problem of group differences arising from individual differences in group make-up
- Requires fewer participants to have good statistical power (because it is a more refined measure of the impact of IVs with a major source of error removed)
- Having the same participants take part in each condition or level of your IV eliminates the problems of individual differences and needing to deal with these via random allocation or matching

49

problems with repeated measure designs

participants may perform differently in each condition based on their prior experiences in the study
can lead to sequencing effects

50

what are sequencing effects?

order effects (practice and fatigue) and carry over effects

51

practice effects

Where completing measures of the same DV or outcome variable multiple times may lead to the participant becoming practiced at the measure.
If the DV is performance based e.g. reaction time then differences improvements may occur simply due to the practice rather than being due to different conditions of the IV

52

fatigue effects

where repeated completion of the IV measure of task may lead to boredom or tiredness

53

what is the solution to the problems of sequencing effects in repeated measures designs?

coutnerbalancing

54

counterbalancing

•In order to control for sequencing effects we can use counterbalancing
•breaking our sample into subsets who will experience the different conditions in different orders
•By then collapsing all the data together again hopefully we can also test for the existence of sequencing effect as well as collapsing across them

55

randomized counterbalancing

sequence of conditions is randomly determined for each participant

56

what are different types of counterbalancing?

randomised, intrasubject, complete, incomplete

57

intrasubject counterbalancing

participants take treatments in ore than one order. May not be feasible with longer treatment sequences

58

complete counterbalancing

all possible sequences of treatment are used. Participants randomly assigned to the sequence
N! = N multiplied by each number below it
rarely used with more than 3 conditions because number of possible sequences (N!) too large

59

incomplete counterbalancing

most common technique
Not all possible sequences are used.
requires a criteria and formula to determine the sequence

60

what is the criteria for incomplete counterbalancing?

• each treatment condition must appear an equal number of times in each ordinal position and
• each treatment condition must precede and be followed by every other condition an equal number of times

61

what are the two carry over effects of counterbalancing?

simple and differential

62

what is the simple carry over effect for counterbalancing?

Where performance on the DV in one condition is contaminated by the effects of the previous condition

63

What is the differential carry over effect for counterbalaning?

Where the carry-over effects of one condition of the IV differ depending on the order in which the conditions are completed.

64

why is counterbalancing a great tool?

It allows you to control for order effects by providing data from a range of different orderings of measurement

65

what happens when time is an IV

participants provide a baseline measurement and then complete measurements at numerous time points during and after the treatment program.

66

what is the problem with time as an IV?

cant counterbalance time

67

what are issues when having time as an IV?

maturation, history, statistical regression, mortality

68

maturation

• Changes due to natural development, natural expected improvement over time etc
• If participants improved over time the question becomes whether or not it is as a function of the IV, or just a natural improvement that would have occurred anyway

69

history (external events)

• External events that affect participants during the study
• Socio-historical-economic changes relevant to outcome/DV
• While participants were taking part in a program to reduce smoking there was a massive price rise in cigarettes

70

statistical regression

• Refers to tendency to move up or down towards the mean over time
• In other words someone who was scoring below or above par on a measure is potentially likely to move towards the mean of that variable over repeated measurements

71

mortality

• Attrition from the study: Not all participants who take part in the first measurement points will remain in the study until the bitter end

72

what are other effects to an experiments validity?

• Experimenter Effects
• Participant Effects
• Situational Effects

73

experimenter effect

• The experimenter as part of the experiment
• Measurement, attributes and expectancies

74

measurement issues regarding the experimenter effect

o problem with equipment or errors in manual recording of data between measurement point or between participants on the part of the experimenter

75

how does one control measurement issues regarding the experimenter effect?

• make researchers aware of making careful observations (training) multiple data records (video camera, computers) or observers have participants make responses on computer

76

attributes effect with regard to experimenter effect

participants responding differentially to different experiments within the study

77

how does one control attribute errors?

use the same experimenter in all treatment conditions (unless the treatment condition interacts with attributes.

78

what are expectancies with regard to the experimenter effect?

are the likely response of participants to the study and its manipulations may lead to subtle differences in the way the experimenter interacts with participants which lead to differences in outcomes

79

what are the two sub categories of expectancies with regard to the experimenter effect?

rosenthal effect and the golem effect

80

rosenthal effect

differential attitude or attention conveyed to participant expected to respond most favourable to the study

81

golem effect

differential attitude or attention conveyed to participant expected to respond least favourable to the study

82

what are controls of the experimenter expectancies?

double-blind method, partial blind technique, automation

83

double-blind method

neither the experimenter nor the research participant is aware of the treatment condition administered to the participant

84

partial blind technique

a method whereby knowledge of each research participant’s treatment condition is kept from the experimenter through as many stages of the experiment as possible

85

automation

the technique of totally automating the experimental procedures, so that no experimenter-participant interaction is required

86

what are problems that come with participant effects?

demand characteristcs, social desirability, hawthorne effect

87

demand characteristics

participant gets an inkling about what the study is aiming to achieve and performs in a way to conform to those expectations

88

social desirability

participant performs in a way they think will be most pleasing to experimenter, or will paint them in the best light

89

hawthorne effect

participant improves or changes performance on the outcome purely as a function of the attention received for being in the study and not as a function of the nature of the manipulations of the IV

90

what are the controls of the participant effect?

single-blind double-blind, deception

91

single blind study

The participant is not made aware of the true purpose of the study or the nature of the group in which they are in

92

double-blind method

neither the experimenter nor the research participant is aware of the treatment condition administered to the participant

93

Deception

o omission of or altering the truth of information given to the participant during a research study
o used when there is no other way to gain the knowledge and risk does not outweigh the benefit of the information
o must keep the false information constant for all participants

94

situational effects

impacts o environmental and timing differences on participants' outcome scores.
Includes time of day, weather, lighting, noises

95

how are situational effects controlled?

keep these constant for all participants or counterbalancing

96

what does control groups allow?

allows you to directly examine the extent to which changes would have occurred without your intervention/s as well as potentially examining a whole raft of other extraneous variables

97

what is a crucial thing to ensure when forming a control group

that it is equivalent

98

if you are to have multiple control groups, what are the conditions of the two groups?

no intervention
placebo intervention

99

what does having multiple control groups counter for?

the hawthorne effect

100

what are the key features of whtin groups designs?

o Minimise individual difference error
o Problems with order effects, practice, cumulative effects, carry-over effects, history, maturation, attrition/mortality
o Can control these things somewhat with counter-balancing, random order variation

101

what are the key features of a between groups design?

o Eliminate problems of WG
o Individual differences come into play
o Ensuring comparability of groups/matching