HowToDesignAResearchStudy Flashcards Preview

CERT > HowToDesignAResearchStudy > Flashcards

Flashcards in HowToDesignAResearchStudy Deck (181):
1

What are the steps to designing a research design? (How Venerial Disease Overtook My Sexy 3 Aunts Royally)

  • Hypothesis
  • Variables
  • Design
  • Operationalizations
  • Manipulation
  • Setting and Participants
  • 3rd Variables (Mediators and Moderators)
  • Analyze Data
  • Report Results

2

Hypotheses should be worded as ________ and should make explicit _____ between groups

statements

comparisons

2

What are the 8 threats to internal validity

  • Maturation
  • History
  • Instrumentation
  • Regression Toward the Mean
  • Testing
  • Selection
  • Mortality
  • Interaction of Selection x Maturation

3

Hypotheses should be worded in the past/present/future tense

present or future tense

3

Describe the internal validity threat: Selection

  • Differences in posttest scores exist because of preexisting group differences
  • possible unequal distribution of subject-related variables (e.g., sex, race, age, intelligence)

4

Hypotheses can/cannot be proven

You cannot prove a hypothesis

4

Describe the internal validity threat: History

  • simultaneously occurring outside events may influence subjects
  • Events that take place during the course of research that influence the posttest scores

5

If you include interactions as part of your hypothesis, you need to be explicit about each experimental condition in the interaction. Give an example.

People who work in homogenous groups will perform better than people who work in heterogenous groups

5

Describe the internal validity threat: Maturation

  • Naturally occurring changes in person over time that influence the posttest scores
  • Permanent: biological growth
  • Temporary: fatigue

6

What question does the Moderator answer

Answers the question when or for whom does the IV have an influence on the DV

6

Describe the internal validity threat: Repeated Testing

  • Ss may become sensitized
  • pretest measurement of the DV may affect the results obtained from subsequent measurements

7

What question does the Mediator variable answer

  • Answers the question how or why does the IV influences the DV
  • often a variable that describes an internal psychological mechanism
  • Often introduced when there is already a strong relationship between IV and DV

7

Describe the internal validity threat: Instrumentation

Changes in the way pretest and posttest scores were measured may cause changes in outcome

8

What should you include when discussing the design of a study

  • Name of the design (be specific)
  • why it is the best design
  • flaws or limitations and how you will attempt to mitigate them
  • determine if btwn or within Ss design (state why)
  • discuss feasibility (economically, logistically, ethically)
  • discuss limitations of feasibility
  • design is usually a 2x2 factorial or 2 btwn-Ss factors

8

Describe the internal validity threat: Regression to the mean

Ss with extreme scores on the pretest will tend to have scores closer to the mean on the posttest

9

Why do you want to use a 2x2 factorial design

  • Economy: the design provides more information from the same amount of work
  • Experimental control
  • Increases Generality of the results
  • A way to investigate interactions among Ivs

*the factorial design allows us to investigate more realistic situations

9

Describe the internal validity threat: Mortality

Loss of Ss during the experiment that is different btwn the  experimental and control groups (e.g., dropping out)

*Differential mortality cannot be ruled out by random assignment

10

Why is it important to test the interaction of Ivs

Because the effect of an IV rarely occurs in isolation

In the real world, many variables operate simultaneously

10

Describe the internal validity threat: Selection x Maturation

Differences between groups that cause changes in the groups at different rates

11

Independent Variable

A manipulation to which participants are randomly assigned (if possible)

11

What is external validity

The extent to which results fo a study can be generalized from the sample to the population or other settings, samples, and variables

12

If you cannot manipulation the IV, what types of studies can you do?

quasi-experimental or correlational

12

Threats to external validity are anything that makes the experimental situation ____

unique

13

For all levels of the IV, you need to _____

identify the operationalization (even for "absence")

*be specific (e.g., give quotes of what experimenters will say, if relevant)

13

_____ experiments have lower external validity; ____ experiments have higher external validity

true, field

14

Why is it important to run a pilot-test

  • Determine if manipulations (IV) have intended effect (i.e., did you manipulate the IV effectively?)
  • to find a balance between impact and control

14

What is ecological validity

whether the research conditions resemble at all the way humans behave in the real world

15

why is it important to include a manipulation check

to determine if manipulations (IV) have intended effect

15

What are the 5 threats to External Validity

  • Selection x Treatment
  • History x Treatment
  • Interferences from Multiple Treatments
  • Testing x Treatment
  • Reactive Arrangements

16

You should specify what type of data will be produced by the IV.

True/False

True.

16

Describe the External Validity threat: Selection x Treatment

results are only generalizable to the specific sample studied, not to the general population (e.g., resarch volunteers tend to be different from non-volunteers)

17

When operationalizing the IV, what should you do to reduce systematic error

  • Carefully operationalize variables so that extraneous variables don’t occur with the IV
  • Randomly assign subjects to conditions

17

Describe the External Validity threat: History x Treatment

Because of events going on in the world, the results of the study can't be generalized to every day life (e.g., doing a study during wartime)

18

What is the dependent variable

The variable being measured; the variable used to assess the affected construct

18

Threats to External Validity: What are the three Interferences from Multiple Treatments (in within-subjects designs)

  • Carryover effects
  • Order Effects
  • Practice Effects

19

When defining the DV, what should you include

  • measurement, scale, type of data produced, meaning of high/low scores, reliability and validity of measurement
  • be unobtrusive in measurement
  • if judges/raters are used, describe criteria for competence and inter-rater reliability

19

Threats to External Validity: Interferences from Multiple Treatments: What are Carryover Effects

Effects of one treatment contaminate the next (e.g., drug A is still in Ss system when he takes drug B)

20

What should you do during the manipulation phase of your study?

  • describe the research setting
  • consent participants
  • describe the cover story
  • reduce sources of error

20

Threats to External Validity: Interferences from Multiple Treatments: What are Order Effects

Treatment A alters participants' response to Treatment B

21

Threats to External Validity: Interferences from Multiple Treatments: What are Practice Effects

Treatment A teaches participants how to respond to Treatment B

22

During the manipulation phase, how do you reduce sources of error when designing conditions?

All conditions should be identical except for the IV

22

Describe the External Validity threat: Selection x Treatment

  • The sample is primed by testing and thus not representative of the population
  • i.e., the pretest increases or decreases the participants' sensitivity or responsiveness to the experimental variable such that the pretested sample is no longer representative of the untested population (to which results are to be generalized)

23

During the manipulation phase, how do you reduce sources of error in terms of experimenter bias?

  • Experimenters should be blind to the hypothesis or subject condition

OR

  • reduce contact btwn experimenter and subjects

23

Threats to External Validity: What are the four Reactive Arrangements

  • Hawthorne Effect
  • Experimenter Expectancy
  • Evaluation Apprehension
  • Demand Characteristics

24

During the manipulation phase, how do you reduce sources of error in terms of practice and order effects?

You should consider practice and order effects.

24

Threats to External Validity: Reactive Arrangements: What is the Hawthorne Effect

Change is a result of being observed and would not really occur in unobserved populations

25

During the manipulation phase, how do you reduce sources of error when describing the study to Ss?

  • Give clear, specific instructions
  • explain the purpose of the study
  • emphasize its importance
  • ensure confidentiality

25

Threats to External Validity: Reactive Arrangements: What is Experimenter Expectancy

The experimenter communicates their expectations to the subjects implicitly or explicitly

26

During the manipulation phase, how do you reduce sources of error in terms of rumors about the study?

Control rumors about the experiment

26

Threats to External Validity: Reactive Arrangements: What is Evaluation Apprehension

Subjects try to avoid negative evaluations

27

What two things should you do at the end of the manipulation phase

  • Manipulation check
  • Debrief participants

27

Threats to External Validity: Reactive Arrangements: What are Demand Characteristics

  • Cues provided by the experimental setting that help the subject to develop naïve hypotheses about the experiment’s purpose and to behave accordingly
  • When participants behave as they think the experimenter would want them to

28

Describe what is done during a manipulation check? Give an example of how you can do a manipulation check.

You make sure the IV was perceived by participants in the intended way (e.g., give a questionnaire at the end of the treatment asking Ss about their experience of the study)

28

What is reliability

How precise is our measure? (the extent to which it is free from random error)

29

What does it mean to "debrief" participants

You ask them how their participation in the study went; ask them if they have any questions

29

What are some ways you can increase reliability

  • Use heterogenous group of Ss
  • Make test items more similar
  • Standardize administrations (and avoid context effects and priming - start with neutral, non-threatening items)
  • Increase the length of your measure
  • Make items and instructions clear (don't use reverse-worded items, give cover story in everyday language)

30

After you determine your population of interest, you can get a representative sample through _________

random sampling

30

Name four ways of assessing reliability

  • Test-retest
  • Parallel Forms
  • Split-Halves
  • Internal Consistency Method

31

When describing your setting and participants, how do you determine how many subjects you need per cell

You do a power calculation. (Jess says you need at least 40-50 Ss per cell)

31

Describe Test-Retest reliability

  • You give the same test twice
  • the correlation between the two scores is the reliability (the reliability coefficient)

32

When deciding who you will and will not include in your study, you should describe __ and __ criteria.

inclusion and exclusion criteria

32

What are the assumptions and limitations of Test-retest reliability

  • assume true score didn't change
  • there are memory effects
  • reactivity (from practice effects)
  • you assume the tests were parallel

33

What are some sample demographics you should describe

gender, race, age, other relevant characteristics

33

Describe Parallel Forms reliability

  • You use two different forms
  • the correlation between the scores on the two forms is reliability

34

If you are using a between-subjects design, you can ____ Ss to groups

randomize

34

What are the Advantages of Parallel Forms reliability

You don't have to worry about memory effects

35

If there are important variables that can be identified and measured and equivalence among groups is a problem (even given randomization) what are other ways of equalizing groups using subject variables

matching or blocking

35

What are the limitations of Parallel forms reliability

  • assume true score didn't change
  • assume your error variances are equal
  • reactivity (from practice effects)
  • it can be difficult to come up with a second test

36

Define and give examples of moderators

  • Moderators can be measured or manipulated
  • Usually a demographic-like variable
    • answers when or for whom
  • (e.g., gender, boring/intersting task, simple/complex task)

36

Describe Split-Halves reliability

  • Administer a single test
  • Use 1/2 of the test for one score and 1/2 for the other score
  • using the correlation, you will get the "split half" reliability

37

How do you test for a moderation effect

By creating an interaction term and seeing if it is significant

37

What are the advantages of uisng Split-Half reliability

You don't have to worry about:

  • stability of the true score (since you only give test once, your score can't change btwn tests)
  • memory effects
  • reactivity (practice effects)

it's cheaper and easier

38

Define and give examples of mediators

  • Mediators can be measured
  • Often a psychological proces variable or an internal psychological mechanism
  • answers the question how or why the IV influences the DV
  • (e.g., anxiety, stress level, self-esteem)
  •  

38

What are the disadvantages of using Split-half reliability

  • The split-halves have to be parallel
  • there are many ways to split the test
  • diminishing returns (as you keep increasing the length of the test, the reliability goes up, but more and more slowly)

39

When analyzing data for a true experimental design, you will usually run an

ANOVA

39

Describe Internal Consistency reliability

compute coefficient alpha/cronbach's Alpha

40

When analyzing data to test for moderation, use a _________

factorial ANOVA

40

What is alpha

the reliability over all split halves

41

When analyzing data to test for mediation, you should run ___, ___, or ___

several regression equations, partial correlation, or path analysis

41

Cronbach's alpha has a range of ___ to ___

0 to 1

42

When reporting results, don't report the main effect when

the interaction effect is significant

42

Standard Error of Measurement measures the ____ of an individuals score (thus related to _____) - that is, how much the score would vary if tested repeatedly (but it is only based on one testing)

Precision, reliability

43

When reporting results, state the results in words with their appropriate _______

statistics

43

The Standard Error of Measurement (SEM) is ____ when reliability goes up

smaller (i.e., you want a small SEM)

44

When reporting results, how should you visually display results?

  • Show a line graph for the interaction or main effect
  • Show a diagram (possibly with correlations) for the mediation effect

44

Validity

The extent to which a measure actually measures what it is meant to measure, and nothing else

45

When reporting results, what alternative explanations (limitations) should you consider

  • limited generalizability (e.g., are you just using college students)
  • IV manipulation is ineffective (i.e., not powerful enough to detect a difference
  • IV operationalization is not effective (i.e., you are measuring something else, another influece diluted your efect)
  • systematic error/systematic differences btwn groups that covary with manipulation
  • confounds

45

If a measure is not _____it cannot be valid

reliable

46

After reporting results, you should give ideas for future _____

research

46

If a measure is not reliable it cannot be _____

valid

47


What is the purpose of a cover story

  • Answers the question: Why are you here?
  • Gives participants the sense that they know what is going on, gives an overall framework in which to interpret the events and tasks called for by the research
  • Keeps pts interested and motivated and to keep them from speculating about what the researcher really wants

*It is best to avoid deception when possible

47

What is face validity

  • Not really validity
  • does a test measure what it superficially appears to measure

48

What components should you include when answering Caryn Block's question on the CERT?

  •         Design
  •         Variables
  •         Hypotheses
  •         Setting and Participants
  •         Operationalizations
  •         Describing Moderation first
  •         Describing Mediation
  •         Alternative explanation of results

48

What is content validity

  • Whether the measure fully represents the domain of the content it is supposed to represent
  • established through a rational analysis of the content fo a test
  • its determiniation is based on individual, subjective judgment

49


What design is usually used on the CERT for the Caryn Block question?


2x2 factorial ANOVA

49

What are the two types of content validity

  • face validity
  • logical validity

50


Hypotheses: Give an example of an explicit comparison that can be made between groups.


People who work in homogenous groups will perform better than people who work in heterogenous groups.

)could also say as compared to)

50

What is criterion validity

The degree of relationship between a measure and a criterion (e.g., 1st yr college GPA, job perf compared to a test you took when you first applied)

51

Operationalizations should be _______. If relevant, give ___ of what research assistants will say.


specific

quotes

51

What are two types of criterion validity

  • Predictive Validity
  • Concurrent Validity

52


When describing the moderation effect, the ___ variable goes on the X axis and the ____ variable goes on teh Y axis.

independent

dependent

52

What is concurrent validity

  • scores discriminate btwn groups that are known to differ at time of testing
  • (data is available at the same time, even if you don't give measures on the same day)

53

How do you describe the mediation effect for the Caryn Block question?


Demonstrate that the mediator is correlated with the DV

In a regressin, demonstrate that the interaction term is significant in predicting the mediator

In a regression with the main effects, interaction effect, and mediator all predicting the DV, demonstrate that the interaction effect is not significant

53

What is predictive validity

  • measures predictor first, then criterion (the criterion becomes available in the future)
    (e.g., getting SAT scores and later getting 1st yr college GPA)

54


What alternative explanations of results do you want to describe for the Caryn Block question?


Any systematic differences btwn groups that covary with manipulations

confounds

history effect (e.g., one room is hotter than the other)

54

What is construct validity

the degree to which the test measure the theoretical construct or trait that it was designed to measure

*You need to demonstrate that

  1. Your measure is related to other measures of your construct
  2. Your measure is related to measures of constructs that are related to your construct
  3. Your measure is unrelated to measures of constructs that are unrelated to your construct

55


What are the 3 desiderata of research?


Control (behaviors)

Realism (Context)

Generalizability (Actors)

55

What falls under construct validity

  • convergent validity
  • discriminant validity
  • multitrait-multimethod matrix

56


What is the 3 horned delimma (desiderata)


If you maximize one or two of the desiderata you will, by definition, minimize the third

56

What is convergent validity

  • measures the same trait by different methods
  • should be correlated with your measurement of the construct

57


A theory can never be ___ / ___ / ___, but rather it is ___ based on data


proven, confirmed, disconfirmed

modified

57

What is discriminant validity

  • different measures of an unrelated construct are uncorrelated with your measure
  • the correlation between different traits measured by the same or different methods (should be smaller than convergent validity)
  • can be used to differentiate your construct from other similar constructs

58


variables are concrete partial representations of ____


constructs

58

What is the multi-trait multimethod matrix

  • A table of correlations that establish both convergent and discriminant validity of a measure
  • used when 2+ traits are being measured by 2+ methods

59


How can hypotheses be worded for categorical variables

 

  • Condition A1 has a positive effect on B as compared to Condition A2.
  • Condition A1 will result in less B than Condition A2.

59


In terms of Internal Validity, by using a more precise test, you will have less ___ ___ the ___ from pretest to posttest

regression towards the mean

60


How can hypotheses be worded for continuous variables

 

  • The higher A, the lower B.
  • As A increases, B decreases.
  • A is negatively related to B.

60


This is the only way to make two or more groups equal on all possible differences

Random assignment

61


What does the moderating variable do

  • Partitions the IV into subgroups that differentially impact the DV
  • Can be subject variables (age, race, and socio-economic status)
  • Can be situational variables (task conditions, environmental conditions)
  • Often introduced when an unexpectedly weak relationship between IV and DV had been found.

61


Random assignment is not random ______

sampling

62

Provide an example of a moderating variable (drawn out)

A image thumb
62


Threats to Internal validity: What 3 factors do not change participants but do change average scores on the posttest

Instrumentation
Regression
Mortality

63

The operational definition of a construct is

the set of procedures we use to measure or manipulate it

63


Threats to Internal Validity: What 3 Non-treatment factors
change participants and their scores

Maturation
History
Testing

64


When considering researchable ideas, mediators may provide ___ ___ for the relationship between the IV and the DV

alternative explanations

64


Threats to Internal Validity: What 3 factors challenge
the assumption of equal groups


Selection
Selection X Maturation
Mortality

65


When considering researchable ideas, the moderator will tell you ___ the relationship is more likely to hold

for what type of people / for who

65


What are 4 problems of lab experiments

  • Some variables cannot be experimentally varied
  • Some variables cannot be ethically varied
  • Some variables occur over a long period of time and cannot be looked at in the context of the lab
  • Some variables might be hard to elicit in a lab

66


Describe the partial mediation model (image)

A image thumb
66


Field experiments have:

High ___
High ___
___ Validity
___ Validity

High realism
High control
Internal Validity
External Validity

67

Describe how mediating variables work (image)

A image thumb
67


Field experiments have ___ control over conditions and ___ random and systematic error than in a lab
Treatment dilution
Treatment confounding
Treatment contamination
Changes in treatments

less, more

Treatment dilution
Treatment confounding
Treatment contamination
Changes in treatments

68


Give an exaple of a model that includes both mediators and moderators (image)

A image thumb
68


What is a factorial design and what does it allow us to do?

  • Looks at how 2 IVs jointly effect a DV
  • Allows us to do two experiments at one time (main effects)
  • Allows us to look at interactions

69


Of the IV, DV, mediators and moderators, which can be manipulated and which can be measured (image)

A image thumb
69


What are some examples of factorial design notation?


 2x2 (2 IVs both with 2 levels)
2x3 (2 IVs - one with 2 levels, one with 3 levels)
2x2x2 (3 IVs - all with 2 levels)

70


What is an example of an interaction (image)

People who receive pay perform better than those that do not when the task they are working on is boring, whereas people who do not receive pay perform better than those who do when the task they are working on is enjoyable.

71


What is random error and what does it do

  • extraneous variables whose average influence on the outcome is the same in all conditions
  • obscures the relationship between the IV and the DV

72

For factorial designs, when creating graphs, the DV goes on the ___ axis, one IV goes on the ___ axis and the other IV __

The DV  goes on the y-axis

One IV  goes on the x-axis

The other IV is plotted

73


What does the main effect of an IV tell us


The main effect of each IV tells us about the relationship between that IV and the DV

74


What does it mean if we have an interaction


An interaction indicates that the effect of one IV is different at different levels of the other IV

75


How can you tell if you have a main effect (using a table)


A main effect of Word Type tells us that more words are recalled when they are concrete

A main effect of Rehearsal Type tells us that more words are recalled when imagery is used
 

A image thumb
76

How can you tell if you have an interaction (using a table)

To calculate interactions we are interested in differences
If the differences are different then you have a two-way interaction

A image thumb
77


What do graphs of main effects look like?

A image thumb
78


What do graphs of interactions look like

A image thumb
79

What is systematic error and what is it a threat to?

  • Systematic error: extraneous variables whose influence on the outcome is different across conditions of the IV
  • Systematic error variables are a threat to internal validity

80


Systematic error results from ____ that cannot be separated from the IV

and distorts the relationship between the ___ and the ___


extraneous variables, IV, DV

81


In a Between-subjects design, each subject is exposed to a ___ and ___ ___ are compared

single condition

group means

82


In a Within-subjects design, each subject is exposed to ___ and ___ ___ are compared

several or all conditions

individual differences

83


What are the advantages of a within Ss design

  • Requires fewer subjects
  • Higher sensitivity
  • Reduces random and systematic error (subject serves as his or her own control)

84


What are the disadvantages of a within Ss design

  • Inter-treatment contamination / Order effects
  •      Practice effects
  •      Sensitization
  •      Carry-over

85


The cover story should be

  • Simple
  • Involving
  • Unrelated to manipulations
  • Inducing the same psychological state (in the experimental and control participants?)

86

Operationalizing the IV:

Describe the relationship between control and impact

  • We need as much control as possible over manipulation so that error is minimized
  • As we increase the impact of a manipulation, we are likely to decrease our control over it.
  • Our IV should have enough impact to show up against background noise, but not so much as to have multiple meanings or alternative explanations

87


What is experimental realism (a way in which manipulation can be realistic)

  • ‘Psychological reality’
  • Manipulation is realistic to subjects and has an impact to them

88

What is mundane realism (a way in which manipulation can be realistic)

 

  • ‘Physical reality’
  • Manipulation is similar to real-life events

89

Demand characteristics are a source of ___ error

systematic

90


What are Strategies for minimizing
demand characteristics

  • A deceptive cover story
  • Collect DVs unobtrusively
  • Expose subjects to only one condition
  • Separate IV manipulation from experimental session
  • Subjects are unaware that they are in an experiment
  • Use Behavioral Measures as DVs

91


How can you Minimize experimenter expectancy

 

  • Experimenter is unaware of condition
  • Experimenter is unaware of hypothesis
  • Multiple experimenters within sessions
  • Multiple experimenters across sessions
  • Experimenter is unaware of condition as long as possible
  • Randomly assign experimenters to conditions
  • Experimenters are trained
  • You can also Minimizing experimenter’s role in the experiment by using media or limiting experimenter-subject interactions

92


What are Alternative methods of establishing equivalence when randomization to groups is not possible

  • Matching (controlling subject variable)
  • Blocking (controlling subject variable)
  • Nested design (controlling contextual variable)
  • ANCOVA (statistical post-hoc procedure)

93


Describe matching in experiments (image)

A image thumb
94

What are the pros and cons of using matching

  • Pro: Equivalence most likely established on matched variable
  • Con: No longer have random assignment - differences are likely to exist after matching on other subject variables
  • Con: Eliminating cases that cannot be matched raises questions about external validity
  • Con: Difficult to choose which subject variable is the only important one

95

Describe blocking in experiments (image)

A image thumb
96


What are the pros and cons of Blocking versus Matching

  • No unconditional random assignment so that other important subject variables are not controlled for, but better random assignment than with matching.
  • Blocking assigns subjects to a few blocks, differences between conditions might still exist after blocking.
  • More subjects are required for successful blocking than for successful matching!

97


Describe Nested designs (image)

A image thumb
98


What are the pros of nested designs

  • Accounts for an important naturally occurring variable that is contextual and is likely to affect the DV
  • Random assignment does exist within each block
  • Equivalence most likely established within each of the blocked groups

99


What is ANCOVA

  • Statistical post-hoc procedure that removes influence of third variable
  • a ‘what if’ analysis: What would happen if all cases scored equally on the covariate?
  1. Randomly assign subjects to conditions OR Create two conditions, not through random assignment
  2. Measure the variable (covariate) you want to control for (e.g.,  pretest score)
  3. Remove influence of this variable statistically with ANCOVA procedure

100


What are the pros of ANCOVA with random assignment

Removes random error due to the control variable, after random assignment

Allows you to quantify the effect of this variable while maintaining equivalence through random assignment
 

101

What are the issues of ANCOVA without random assignment

  • Attempts to remove systematic error due to the control variable, to establish equivalence between groups
  • Groups are not equivalent on many other variables
  • Can’t use ANCOVA to establish equivalency between groups post hoc

102


What is internal validity

  • the extent to which conclusions can be drawn about the causal effects of one variable on another
  • Allows us to make the statement We can make the statement: IV causes DV

103


In correlational research, the IV is ___ not ___

measured

manipulated

104


What are some reasons why you would do correlational research, measuring rather than manipulating the IV

 

  • Variable of interest cannot be manipulated
  • It is unethical to manipulate the variable
  • It is not practical to manipulate the variable

105

What are the three criteria for establishing causality:

  • Association (correlation)
  • Time Priority (cannot be established if IV and DV are measured simultaneously)
  • Non-spurious Relationships (A third variable causes the relationship between IV and DV. The correlation between IV and DV is entirely determined by the third variable.)

106


Establishing Causality: What is a spurious relationship? Give an example.

 

  • A third variable causes the relationship between IV and DV. The correlation between IV and DV is entirely determined by the third variable.
  • IV and DV share no variance after removing the covariance of a third variable.

(e.g., The rel btwn the number of Storks (IV) and Number of Babies (DV) is caused by population density (EV))

107


Establishing Causality:What is full mediation? Give an example.


The third variable mediates the relationship between IV and DV. The correlation between IV and DV is entirely determined by the third variable.

e.g., The rel btwn Counseling (IV) and Well-being (DV) is mediated by Social support (EV)

108

Establishing Causality: What is Partial Explanation/Isolation? Give an example.

The third variable influences the relationship between IV and DV. But it does not entirely determine the relationship

 

e.g., the rel btwn Satisfaction (IV) and Performance (DV) is partially explained by Rewards (EV)

109


Establishing Causality: What is Partial mediation? Give an example.

The third variable mediates the relationship between IV and DV. But there is also a direct relationship between IV and DV.

e.g., the rel btwn Counseling (IV) and Well-being (DV) is partially caused by Social support (EV), but there is also a direct rel btwn the two

110


What is validity


Does the measure reflect the construct we want to measure and no other systematically varying constructs

111


A measure can’t be v___ if it is not r___

valid

reliable

*Reliability is a necessary, but insufficient condition for validity

112

113


What type of wording should you avoid when writing test items

 

  • Ambiguity
  • Jargon
  • Length
  • Double-barreled
  • Leading
  • Loaded
  • Threatening
  • Over-demanding
  • Over-specificity
  • Relevance

114


What is the Multitrait Multimethod Matrix (construct validity)


A table of correlations that establish both convergent and discriminant validity of a measure