Test 3 Flashcards
Validity
Validity: the extent to which empirical evidence and theoretical rationales support the adequacy and appropriateness of conclusions drawn from some form of assessment.
Construct validity:
Construct validity: the ability of a measurement tool or test to accurately measure what it’s supposed to measure.
Internal validity:
Internal validity: the extent to which a piece of evidence supports a claim about cause and effect.
External validity:
External validity: the extent to which the findings of a study are relevant to subjects and settings beyond those in the study
Increasing internal validity often means
sacrificing external validity
And vice versa
External validity asks the question of generalizability:
To what: situations, people, settings, and variables can an effect be generalized?
Ask a research question about a population. ->
Take a representative sample from the population. ->
Answer the research question by studying the sample. ->
Generalize results from the study of the sample to population.
“What is true of the sample is true of the population”
Ask a research question about a population. ->
Take a representative sample from the population. ->
Answer the research question by studying the sample. ->
Generalize results from the study of the sample to population.
“What is true of the sample is true of the population”
The Artificiality Criticism
Scientific experiments are useless because they are artificial and not like “real life”
This assumes that “Real Life” is a better setting for experiments than a controlled laboratory setting.
Why is that a poor assumption to make?
Scenario Number one:
Experiment generalizes to real world.
If scenario one is the goal, then EV is required.
Scenario number two:
Real world inspires Experiment.
If Scenario two is the goal, the EV is not required.
Harlows Monkeys
In the 50s it was believed that a baby’s attachment to their mother was because they fed the baby.
Hunger reduction model of maternal love.
Are the infant monkeys representative of all infant monkeys?
Does the experimental setting approximate real life?
Is the subject’s behavior is influenced by the experimental setting?
Then FAIL: little external validity.
Harlows Monkeys
In the 50s it was believed that a baby’s attachment to their mother was because they fed the baby.
Hunger reduction model of maternal love.
Are the infant monkeys representative of all infant monkeys?
Does the experimental setting approximate real life?
Is the subject’s behavior is influenced by the experimental setting?
Then FAIL: little external validity.
Why conduct controlled experiments if the results don’t/can’t apply to the real world?
Because:
There are inherent benefits to doing these things in and of themselves.
- Ask whether something can happen rather than does happen
- prediction may specify that something should happen in the lab, so test it out.
- Demonstrate the power of the phenomenon by showing that it occurs in a contrived setting.
- Use the Lab to produce conditions that otherwise do not exist in the real world.
Experimental findings not intended to be generalized can still be worthwhile in theory development!
Harlow’s experiment failed to meet the requirements of external validity.
Harlow had no intention of generalizing these results.
He showed that the hunger reduction model didn’t hold up as it should.
Theoretical proposition:
Theoretical proposition: Parent approval/disapproval shape childrens grammatically correct speech.
Prediction: The observed parents ought to do this (disconfirmed by study).
Subjects are of interest precisely because they are unrepresentative.
Past theories and evidence → current theory and evidence → predictions
Past theories and evidence → current theory and evidence → predictions
“BREAKTHROUGH” HEADLINES
• Media headlines often present research findings as “major breakthroughs” . May spread misinformation and exaggeration
CONNECTIVITY PRINCIPLE
- Science obeys the connectivity principle:
- A new theory must account for new evidence as well as previously established empirical facts
• May explain observations (old and new) in a different or unconventional
way
. But it must explain them all to be a true scientific advance • I.e., the new explanation must make sense given what we already know • Cumulative process of science
BEWARE OF VIOLATIONS OF CONNECTIVITY
• Pseudoscience often dismisses previous data as irrelevant in light of
a new “breakthrough” theory • Novelty and “radical departures” are emphasized
• Pseudoscience relies on long-held “wisdom” AND novelty and
“radical departures”
“GREAT-LEAP” MODEL
People tend to believe that scientific advances occur with “great leaps”
Reality: progress and setbacks
Gradual synthesis:
• Progress occurs as the community of scientists gradually begins to agree that the evidence supports one explanation over another
. Case studies
- In depth research about one specific person or group
* Not representative
• Correlational studies
- Implies relationships between variables • Does not determine cause and effect
- Limitations in regard to directionality and third variables
• Quasi-Experiments
• Like a correlational study in that variables aren’t manipulated • Like an experiment in that one preexisting variable is used to group
individuals and is treated like an IV
• Experiments
- Can determine cause and effect
- Manipulate one variable (IV), control all other variables, and measure another variable (DV)
- Often utilizes random assignment
- Issues of confounding variables and validity
FLAWS IN RESEARCH
• Flaws and limitations, threats to internal/external validity • Ambiguity of interpretation of data from any one study
Principle of converging evidence: Support for theories comes about when the preponderance of evidence from many different types of studies point to the same answer
Principle of converging evidence: Support for theories comes about when the preponderance of evidence from many different types of studies point to the same answer
WHAT CONSTITUTES CONVERGING EVIDENCE?
• Two key factors:
The results of many studies consistently support one theory
- They collectively eliminate competing theories
- When evidence from a wide range of studies all point in a similar direction, evidence has
Converged
SCIENTIFIC CONSENSUS
- Methods & Convergence
- Should expect many different methods to be used in research
- Fields of psychology should be careful not to become over-reliant on one method of study
- The research process often proceeds from weaker methods to more powerful ones
- Case study correlation
• Meta-analysis:
• Meta-analysis: statistical method for combining results of many Studies
META-ANALYSES
• A statistical method of combining findings from a large number of
studies of the same topic to determine an overall pattern
- Procedure:
- define the effect of interest .
- search the published literature • code the characteristics of the studies you find
calculate effect size
. E.g., correlation coefficient, standardized mean difference, odds ratio
Meta Analyses
Two reasons why Mas are better than primary studies:
Condense volumes of research into a single meaningful investigation.
Results are more reliable than a single primary study.
Problem: the “file-drawer problem”
Non-significant results are less likely to be published.
Solution: failsafe
The number of non-significant results need to change the conclusion of the MA.
Effect size:
Cohen’s d One measure of effect size. Effect size represented in numbers. Small d=.20 Moderate d=.50 Large d=.80 Effect size represented in distributions.
Multiple Causation and interactions.
The search for the magic bullet.
What is the Magic Bullet?
The (one and only, all-purpose) cause of something.
But more psychological phenomena are more complex than that?
Multiple causation
When not one but many factors/variables contribute to a psychological phenomenon.
Multiple causation is different from converging evidence.
CE: when multiple studies point to one theory.
Many phenomena are influenced or “caused” by more than one factor.
What causes breast cancer?
Biological factors: Genes Family history Age Environmental factors: Diet Exercise Obesity Alcohol consumption Exposure to carcinogens
But factors also interact with one another
Family history and alcohol consumption, age and obesity, ect.
Interactions
Sometimes these multiple causes interact with one another.
Interaction: the joint effect of two or more variables on another variable
Interaction effects occur when variables have not only separate effects but also combined effects on an outcome.
The smiley face example:
What this demonstrates:
The two factors combine in such a way that, to earn the highest tip, women should include a happy face and men should not.
Happens with many different constructs psychology is interested in.
Problem: the more factors you measure as IVs or predictors, the harder it is to interpret the effects on the DV.
Types of variables that can be tested
Last example dealt with categorical IVs ie man or woman.
Can also test for interactions with continuous IVs
Social media use
Question: do people who use social media a lot have less face to face time.
Positive correlation r=.12
Question: does this change depending on personality traits.
People who are more introverted would get more face to face time with facebook use.
Question: does this change depending on personality traits.
People who are more introverted would get more face to face time with facebook use.
Some of Harlow’s experiments on monkeys tested which of the following.
The hunger reduction model of maternal love.
Advances in science occur in fits and starts, progress and setbacks. This concept is known the _____ model.
Gradual synthesis model.
An interaction occurs when there are combined effects of two or more variables on another variable.
True
