ResearchDesignFlaws Flashcards Preview

CERT > ResearchDesignFlaws > Flashcards

Flashcards in ResearchDesignFlaws Deck (24):
1

What are the 10 big design flaws

  • no control group
  • using the wrong statistical test
  • misuse of correlation
  • misuse of ANCOVA
  • Misuse of Standard Error of Measurement (SEM)
  • Regression Toward the Mean
  • Sampling Problems
  • Using Non-Equivalent Groups
  • Confusing Discriminant and Convergent Validity
  • Poor Reliability of Measurement/Confusing Reliability with Validity

2

What threats are present if you don't use a control group

  • history
  • maturation
  • testing (if a pre-test design)
  • instrumentation (if measurement could be changed from pre- to post- test)
  • threats to internal validity present (resulting low generalizability)

3

Statistical tests that use categorical variables

  • chi-square
  • logistical regression

4

Statistical tests that use continuous variables

  • t-tests
  • ANOVA
  • regression

5

What are the two ways you could misuse correlation

  • confusing correlation with causation
  • wrongly concluding there is no correlation between two variables

6

What type of relationship do you have when two variables are associated only because they are influenced by a third variable?

Spurious relationship

7

You should only use ANCOVA to establish equivalency among groups if you used _____ assignment

random

8

Standard error of Measurement is a measure of _____

precision

9

Standard error of Measurement is NOT a measure of _____

validity

10

You want Standard Error of Measurement to be small/large

small

11

What is regression towards the mean

If you only look at extreme scores, when you re-test the scores are likely to be less extreme

12

What are two sampling problems

  • trying to generalize a sample of convenience to the entire population
  • trying to generalize a specific population to a much larger population

13

If you must use non-equivalent groups, you should at least use a ____ for comparison

pre-test

14

What is Convergent Validity

measure of the same trait by different methods

should be correlated

15

What is Discriminant Validity

the correlation between different traits measured by the same or different methods (should be smaller than convergent validity)

16

When using a measure, we want a Cronbach's Alpha of at least

.70, or at least .80 if there are 9+ items

17

How can you increase reliability of a scale

Add additional items

18

When you add additional items to a scale, you increase ___ but not ____

reliability, validity

19

How should you examine rates of attrition?

Look to see if drop-out rates differ in some (systematic) way

20

What has happened when you have poor operationalization of variables

your operationalization does not seem to really capture the actual construct or it capturs other things as well as the construct

21

What do you have when the experimental manipulation does not feel real the the subjects

low experimental realism

22

What do you have when the experimental manipulation does not mirror the real world

low mundane realism

23

What words describe badly written measure items

  • ambiguous
  • double-barreled
  • leading
  • emotionally loaded
  • over-demanding
  • over-specific
  • confusing (e.g., reverse-worded)

24

It is difficult to generalize results if your scores have high ___ and ____

  • variance,
  • standard deviation

(i.e., it is hard to generalize results if items are spread out and there is little consistency)