Exam 3 Lecture 6 Flashcards
Concrete or abstract concept
Construct
Deconstruct the concept/construct into components
Items/variables
Stats Steps
- Have a question
- Make a plan
- Gather data
- Describe and visualize
- Have a question.
Nail down the details/figure out exactly how you are going to ask it.
- Concrete or abstract concept = construct
- Deconstruct the concept into its components = variables
- Find a abalone between simple and specific variables; make sure variables are sufficient
Determine exactly how you want it answered
- Qualitative (open-ended or categorical)?
- Qualitative/categorical -> ordinal, nominal, binary?
- Quantitative -> discrete, continuous?
Check your ideas!
- Think about your questions carefully
- Think about your answers carefully
- Work to limit noise and errors
- Construct (face, content, criterion) validity?
Common sense goes a long way!
- Make a plan.
Develop a protocol: it’s like a recipe; it needs step-by-step instructions
- Observational/experimental? Cross-sectional/longitudinal (retrospective/prospective)?
- Blinding? Deception? Masking? Considerations of bias?
Find a sample
- Make sure your sample represents your population
- Make sure your sample is large enough
Check your protocol!
- Look at your data carefully!
- Make sure your protocol is reproducible
- Check for bias in your sample
- Study (internal/external) validity!
- Gather data.
- Go ahead and get yourself some data.
- Make sure the data you get is directly related to your construct
- From the very beginning - organize your data (Excel!)
Check your data!
- Datasets!
- Cleaning!
- Raw data & transformations
- Be consistent! Study drift is a real thing!!
- Spot check for missingness/errors
- Describe and visualize.
- What’s the mean, mode, and/or median?
- What’s your range and standard deviation?
- Are your data normally distributed?
Skew
Kurtosis
Outliers - Graph it!
Mandatory Data Check!
- Non-normal data can’t be analyzed the same way!
- Outliers can mess everything up!
- Graphs can help you pick a statistical approach
For the data consumer:
1. What was their question?
How was it asked?
- Did they deconstruct the concept properly into its component?
- Were their questions simple and specific?
- Was their approach to getting an answer complete and sufficient?
What options were given for answers?
- Did they collect qualitative (open-ended or categorical) or quantitative data?
- Were their responses reasonable?
Check their ideas!
- Do you understand what the researchers were studying?
- Would you use these questions/response options or methods to study it?
Reframed for the data consumer
2. Did they have a clear plan?
Was there a protocol (procedure), with all necessary detail?
- Was the study observational or experimental?
- Was the study cross-sectional or longitudinal (retrospective/prospective)?
- Was there blinding?
Did they describe their sample, with all necessary detail?
- Did the sample seem reflective of the population of interest?
- Does the sample seem large enough? Too large?
Check their protocol!
- If you wanted to do the study yourself, could you? What would you do differently?
- What do you really think of their sample? Representative? Good size? Was their protocol reliable? Consistent?
Reframed for the data consumer
3. What kind of data did they end up with?
Objective? Subjective? Messy/dirty/noisy/error-prone?
Did they report about cleaning/quality checking (sometimes called processing) the data?
Check their data!
- How much was missing? Did they exclude a bunch of stuff (fancy way of saying error)? Did they explain why? Do you agree with what they did and why?
- Are there possible confounding variables?
- Do the data seem generalizable?
Reframed for the data consumer
4. Describe and visualize.
Have they told you the mean, mode, and/or median?
- Do they report the range and standard deviation?
- Do they specify that their data distributions are normal?
- Are their graphs informative? Are they intuitive?
Mandatory data check!
Look at the descriptive!
- Are the data normally distributed?
- If not, what do you see/suspect? Skew, kurtosis, outliers?
Look at the graphs carefully!
- Does the type of graph make sense?
- Do the axes make sense?
Evidence versus proof
A study, experiment, big data analysis… provides evidence but does not prove anything.
You need multiple piece of evidence to work towards proof.
- In science, that often takes decades.
- It is pretty much impossible to get sufficient evidence for a fad in a short amount of time.
- Sometimes they work out, but that’s mostly luck. Most often fads fade because the evidence does not hold up.
Why only evidence, not proof?
- All studies have the potential for bias
- Introduced by the design, sample selection, experimenter, participant - Statistics are ESTIMATES!
- You are trying to use a bit of info to predict universal truths
- Yeah, that’s gonna take a bit of time - Statistics are PROBABILITIES!
Statisticians have tools that allow them to assess the quality of data that they are getting. This is done when?
- BEFORE they analyze their data
- BEFORE they look for insight/results/conclusions
- BEFORE they use inferential statistics
Inferential statistics
A branch of statistics (like descriptive statistics) that helps draw conclusions/generalizations about a population by analyzing data from a sample
