Final Flashcards
Difference between a sample and a population
Sample: part of the population being studied with the goal to make a claim about the population as a whole
Population: the group as a whole
Compare the purpose and interpretation of descriptive statistics and inferential statistics
DS: make precise statements that summarize the data: central tendency (mean, median, and mode) - how participants scored overall and variability (SD and variance) is how widely the distribution scores spread
IS: a way to help us infer whether a difference in sample means reflects a true difference in the population means
•determines if something is statistically significant: if the difference between mens an impact of the IV or random error
Null vs Research Hypothesis
Research Hypothesis: the means are not equal between the CG an EG/variables are related (r≠0.00)
•IV has an effect on DV
•very unlikely it was caused by random chance
Null: the variables are unrelated in this population (r=0.00)
•IV has no effect on DV
•a precise statement: population means are exactly equal or the correlation is exactly 0 (not possible with a research hypothesis)
•random chance caused any difference between groups
•start inferential statistics by assuming this is true and seeing if we can rule it out with data
These hypothesis are the only two possibilities in the population
•null is rejected when there is a very low probability that the results could be due to random error (significant)
ex:
RH: people will feel less socially connected if they have their phones for directions
NH: people feel just as socially connected if they have their phones
•its pass/fail: there isn’t higher or lowe
What is statistical significance
An inferential statistic to determine if results are due to the IV rather than random chance
•If the null hypothesis is true, this is the likelihood that we would see a difference between means that is large (not just by chance)
•a matter of probability: the probability that the null hypothesis can be rejected
•p < .05 (can’t publish if its higher than .05)
•the smaller the p value, the more likely the statistical significant
•if there’s statistical significance, the null hypothesis is rejected
•its pass/fail: there isn’t higher or lower significance (thats the job of effect size)
How is statistical significance determined
1. Capture a statistic that captures the effect
2. Refer to the matching sampling distribution for comparison
3. Make a decision about the null (is our statistic rare enough to consider it significant)
*a .05 significance level says that you will conclude there is an effect in the population when there isn’t (if 5/100 times that the test is repeated on different samples of that population)
•most likely to brain results when u have a large sample size and large effect size
T test:
•statistic used for two group designs (and a few other specific cases)
*repeated measures and between group versions
•not suitable for more than two groups or complex designs
•DV must be measured on interval or ratio scale
Compare “statistical significance” with “effect size.”
Effect size: the strength of the relationship between two variables
•it is possible for effects of any size to be statistically significant when using large sample sizes
•if there is a small effect size we might nor really care about statistical significance
Discuss multiple challenges to generalizing the results of a single study.
Characteristics of the sample
•uni students: more educated, WEIRD population, not high generalizability to whole world
•volunteers: how the study is advertised will attract different kinds of people
•sex of participants: results of prominently one gender cannot be generalized (should replicate)
•culture: now with more diversity in uni, we have more external validity by seeing a diverse range of people
Characteristics of the study situation
•experimenter: the influence of the experimenter can lower generalizability *solution: use 2 or more experimenters
•pretests: may limit generalizability to populations that don’t receive a pretest
•beyond the lab: conducting research in a lab has the goal of high internal validity, external validity is stronger in field research
*this is changing as we see field and lab often coming to same conclusions
Characteristics of different populations
Confidence intervals: how confident you are that the data will fall in that range
•clue that difference is statistically significant for two means: they do not overlap
•can be another way to reject the null
•the size of an interval is related to the size of the sample and the confidence level
•as sample size increases, the confidence interval narrows because the population value is being estimated more precisely
•confidence intervals around effect size estimates offer idea about who results would turn out if we replicated the study
- Conclusion validity: conclusions drawn about the relationship among variables are correct and reasonable given the data
•requirement for research
•helpful tools: effect sizes and statistical significance
•avoid over/under claiming
•ex: claiming cause/emphasizing limitations too much - External validity
•difficult to make claims outside population - Primary participants (aren’t always randomly assigned)
•convenience samples: delivering animals, uni students and their children - Importance of replications
- direct replication: attempt to replicate the study as closely as possible to see if the same results are obtained
*very important in determining if a finding can be generalized to other populations
•replication with extension also possible
•just because something doesn’t replicate doesn’t mean its disproven (many other explanations) ex: mozart thing - conceptual replication: use of difference procedures to replicate a finding
•the same IV is manipulated in a different way or the DV is measured in a different way
•if a finding is concluded in a study that is differently operationally defined, it is generalized beyond its original setting/operational definition
•NOT substitute for direct replications (they have the potential to create type 1 errors)
•its possible to just throw out study and use OD with findings until the ordinal effect is replicated
Suggestion
•use direct replication first to ensure generalizability, then use alternative ODs in conceptual replication to develop a theory
•type 1 error: the original results were a fluke
•rich understanding comes from replication
Argue for and against the use of convenience samples in basic research.
Convenience samples
pros: sometimes needed due to the demand of the study (difficult to get babies - uni students = access to more participants)
cons: need to be cautious when generalizing claims, not the best to use this sample, but can if needed
Explain when random selection is necessary and when it is not.
Random selection is necessary when the population must have an equal chance of being chosen, if your study involves satisfaction levels, or you’re striving for high generalizability
Distinguish between basic and applied research.
Basic: theory, looking for cause and effect, no pop in mind
Applied: specific environment /event making statement about a specific population (psych does applied research)
Summarize the controversy over the 2011 Canadian census.
- Diederk Stapel (fraud)
•widespread data fabrication with 58 papers retracted
•reported in NYT - Daryl Bem: ESP
•published article showing evidence of ESP in top journal JPSP
•9 experiments: people were able to anticipate future events (pre-cognition)
•widely ridiculed/criticized, but he wasn’t malicious
*removed outliers, partial publication, figure out what worked then write hypothesis after, change operational definitions with each replication - False positive psychology
•Simmons, Nelson, and Simonsohn published paper suggesting nothing is significant
•spotlight on questionable research practices (QRPs)
•offered some explanation for how Stapel and Bem could get published
Census: Harper announced long form was optional and only 20% of the population answered - the only people that do are the ones who have the time
That has now changed
Discuss pressures on science and scientists that led some questionable research practices to become normal in the past.
Can only publish statistically significant effects that are novel
•must publish early in career or don’t get tenure/get fired
•huge pressure to find statistically significant results – questionable research
How we have responded to replication crisis since 2011 and how the disciplinary reform is changing again
Origin of the replication crisis: you could only publish statistically significant results, and you must polish early in your career
Overcome: journals are now accepting direct replication studies, changing job descriptions, open badges awarded for different kinds of research
Reflect on the importance of learning about current replication controversies in the context of your psychology degree.
Being conscious when using archived data,
Identify multiple strategies being considered and implemented in our discipline to help us find truth
Replication studies encouraged rather than discouraged
