Flashcards in Scientific Investigation Deck (34):

1

##
5 Steps of scientific investigation (Kazdin, 2003)

OSCAR

###
-Originate (formulate) a testable hypothesis

-Select a research method and design the study

-Collect the data

-Analyze the data

-Report the findings

2

## O: Originate (formulate a testable hypothesis)

###
-hypothesis

-null hypothesis

-alternative hypothesis

-IV

-DV

3

## hypothesis

### a tentative statement about the relationship between variables

4

## Null hypothesis (Ho)

### hypothesis that specifies that there is NO difference between conditions or groups in the experiment on the dependent measures of interest

5

## Alternative hypothesis (Ha)

### hypothesis that specifies that there IS a difference between conditions or groups in the experiment on the dependent measures of interest

6

## IV

### variable (construct, experimental manipulation, intervention, factor) that is manipulated in the study

7

## DV

### the measure designed to reflect the impact of the IV, experimental manipulation, or intervention

8

## S: Select a research method and design the study

###
-true experiment

-quasi experiment

9

## true experiment

###
– must meet conditions (Kazdin, 2003):

random assignment of participants to conditions

maximum control of IVs/condition of interest

investigator can include alternate conditions (i.e., treatment and control conditions)

investigator can control possible sources of bias within the experiment that permit the comparison of interest

i.e., randomized control trial is a good way to see if an intervention is effective

10

## quasi-experimental

###
conditions of the experiment are approximated; restrictions are placed on some aspect of the design (Kazdin, 2003):

e.g., can’t use random assignment to groups

e.g., convenience sample

11

## statistical significance

###
-The criterion is used to evaluate the extent to which the results of a study (e.g., differences between groups or changes within groups) are likely to be due to genuine rather than chance effects.

-The level that demarks statistical significant (called alpha and designated with a Greek letter α) is completely under the control of the researcher

-Norms for different fields exist

-For example, .05 is generally used in educational/psychological research.

12

## What does .05 mean?

###
-The level of statistical significance is the level of risk that the researcher is willing to accept that the decision to reject the null hypothesis may be wrong by misattributing a difference to the hypothesized factor, when no difference exists.

-In other words, the level of statistical significance is the level of risk associated with rejecting the null hypothesis.

13

## Selecting α = .05 indicates that the researcher

### is willing to risk being wrong in the decision to reject the null hypothesis 5 times out of 100, or 1 time out of 20.

14

## Type I error

###
Decision: Reject the null hypothesis

Reality: there is no significant effect

*probability of rejecting the null hypothesis when it is true (α)

False Positive

15

## Type II error

###
Decision: Fail to reject the null hypothesis

Reality: there is a significant effect

*probably of accepting the null hypothesis when it is false (beta)

False Negative

16

## Power

###
-Probability of rejecting the null hypothesis (that there are no differences) when, in fact, that hypothesis is false

-Alternatively, detecting a difference between groups when, in fact, a difference truly exists

-Put another way: the extent to which an investigator can detect a difference when one exists

17

## Power analysis

###
-a way of deriving at the ideal sample size number

- to increase power, increase sample size

18

##
Four concepts involved in power analysis

SEPS

###
-statistical significance

-effect size

-power

-sample size

*once three are known, the fourth can be determined

19

## Effect size

###
-The measure of the strength of the relationship

-Cohen's D

20

##
_____ small effect size

_____ medium effect size

_____ large effect size

(Cohen, 1998)

###
.2

.5

.8

21

## Common measures of effect size

###
-Cohen's D

-Pearson's r

-eta-squared and partial eta-squared

22

## Three different types of control group

###
1. no treatment control (unethical in psychotherapy research)

2. waitlist control

(group is withheld from treatment for a period of time, usually time it takes treatment group to complete pre-test, treatment, and post--test)

while waiting, the control group completes pre-test and post-test

3. nonspecific treatment control

(substitute of pseudo-intervention that involves subjects in some sort of treatment experience (not considered a mechanism of change)

23

##
5 types of data that can be included in research

IPPAD

###
-Interview

-Physiological data

-Psychological report

-Archival data

-Direct observation

24

## Continuous vs. categorical variables

###
-continuous: interval and scale

-categorical: dichotomous, nominal, ordinal

25

## Effectiveness study

### actual patients, actual population

26

## Efficacy study

### RCT, random sampling from population, discreet dx

27

## Validity: intent to treat analysis

###
-Keep everyone in, but you use the last score of those who attritioned for the data

-Can also do chi square (like a T-test, except with nominal variables) on demographics to compare the people who completed vs. those who didn’t to account for more systematic variance

-Intent to treat is contrasted with completer analysis (or per-protocol analysis), which only takes into account those who complete the study. Subjects who have not completed the measures (who dropped out of treatment before posttreatment or follow-up assessment) are omitted from the data analysis. However, this approach does not show the practical value of what is being measured (e.g., a new drug)

28

## What kind of test can you run to double check random assignment?

### Chi-square

29

## Drop-our rates of therapy research

###
-Men (Leong & Zachar, 1999)

-Low SES more likely to drop-out (Leong & Zachar, 1999)

-Ethnic minorities are more likely to drop-out (Leong & Zachar, 1999)

-People over 65 (Leong & Zachar, 1999)

30

## Luborsky's Therapist Allegiance Effect (Luborsky et al., 1999)

###
-whatever orientation of the researcher tends to be the one that is significant

-APA – in order for therapy to be considered evidenced-based, research needs to be conducted by at least two independent researchers

31

## anytime RCT is used, researchers must

### randomly videotape to adhere to APA guidelines

32

## When doing an experiment or quasi-experiment, you always have to run ____

### a manipulation to check to see if your variable changed anything

33

## Pearson correlations can only be used with what type of data?

### interval and ratio

34