What are the 10 big design flaws

no control group

using the wrong statistical test

misuse of correlation

misuse of ANCOVA

Misuse of Standard Error of Measurement (SEM)

Regression Toward the Mean

Sampling Problems

Using NonEquivalent Groups

Confusing Discriminant and Convergent Validity

Poor Reliability of Measurement/Confusing Reliability with Validity
What threats are present if you don't use a control group

history

maturation

testing (if a pretest design)

instrumentation (if measurement could be changed from pre to post test)

threats to internal validity present (resulting low generalizability)
Statistical tests that use categorical variables

chisquare

logistical regression
Statistical tests that use continuous variables

ttests

ANOVA

regression
What are the two ways you could misuse correlation

confusing correlation with causation

wrongly concluding there is no correlation between two variables
What type of relationship do you have when two variables are associated only because they are influenced by a third variable?
Spurious relationship
You should only use ANCOVA to establish equivalency among groups if you used _____ assignment
random
Standard error of Measurement is a measure of _____
precision
Standard error of Measurement is NOT a measure of _____
validity
You want Standard Error of Measurement to be small/large
small
What is regression towards the mean
If you only look at extreme scores, when you retest the scores are likely to be less extreme
What are two sampling problems

trying to generalize a sample of convenience to the entire population

trying to generalize a specific population to a much larger population
If you must use nonequivalent groups, you should at least use a ____ for comparison
pretest
What is Convergent Validity
measure of the same trait by different methods
should be correlated
What is Discriminant Validity
the correlation between different traits measured by the same or different methods (should be smaller than convergent validity)
When using a measure, we want a Cronbach's Alpha of at least
.70, or at least .80 if there are 9+ items
How can you increase reliability of a scale
Add additional items
When you add additional items to a scale, you increase ___ but not ____
reliability, validity
How should you examine rates of attrition?
Look to see if dropout rates differ in some (systematic) way
What has happened when you have poor operationalization of variables
your operationalization does not seem to really capture the actual construct or it capturs other things as well as the construct
What do you have when the experimental manipulation does not feel real the the subjects
low experimental realism
What do you have when the experimental manipulation does not mirror the real world
low mundane realism
What words describe badly written measure items

ambiguous

doublebarreled

leading

emotionally loaded

overdemanding

overspecific

confusing (e.g., reverseworded)
It is difficult to generalize results if your scores have high ___ and ____

variance,

standard deviation
(i.e., it is hard to generalize results if items are spread out and there is little consistency)