Session 7 Flashcards Preview

Research Methods in Counseling > Session 7 > Flashcards

Flashcards in Session 7 Deck (120):
1

The two types of Experimental Designs are...

1. Causal
2. Correlational

2

Causal Experiments mean that...

one variable directly or indirectly influences another

3

Correlational Experiments mean that...

changes in one variable accompany changes in another

4

What are the Three Criteria for Causal Relationships?

1. co-variation between X and Y variables
2. Temporal sequence, X occurs before Y variable
3. Control of third (extraneous or confounding) variables

5

When the behaviors and characteristics of the researcher influence the reactions of participants, it is called...

Researcher Error

6

When the personal characteristics and experiences of participants influence their responses, it is called...

Participant Error

7

When aspects of the environment influence scores, it is called....

Environmental Error

8

When aspects of the stimuli presented (the IV) or measurement procedure employed influence scores, it is called...

Measurement Error

9

What is the definition of Third Variables?

1. Any variable that can potentially influence results, but is NOT intended by the researcher
2, Any variable that may compete with the IV in explaining the outcome of the study (DV)

10

How do Third Variables influence research?

1. By influencing the scores & thus the mathematical relations produced.
2. Leads to errors when interpreting and generalizing the study, because the variables that we think are operating are not those that are really operating

11

What is a Dis-simulation effect?

Third variable result of increase in scores

12

What is a masking effect?

Third variable result of decreasing scores

13

What is the definition of an Extraneous Third Variable?

1. Any variable that can potentially influence results, but is NOT intended by the researcher
2, Any variable that may compete with the IV in explaining the outcome of the study (DV)

14

How do Extraneous Third Variables influence research?

1. By influencing the scores & thus the mathematical relations produced.
2. Leads to errors when interpreting and generalizing the study, because the variables that we think are operating are not those that are really operating

15

What is a masking effect?

Third variable result of decreasing scores

16

How does an Extraneous Third Variable fluctuate?

1. unsystematically, randomly, no consistent pattern
2. systematically, consistent pattern

17

What are the two types of Third Variables?

1. Extraneous Variables
2. Confounding Variables

18

What is a Confounding Variable

1. an extraneous variable the DOES compete with the IV in explaining the outcome of the study
2. Systematic variation in an extraneous variable

19

List FOUR examples of Confounding Variables

1. Hawthorne (Attention) Effect
2. John Henry Effect
3. Placeo Effect
4 Demand Characteristics

20

List FOUR examples of Confounding Variables

1. Hawthorne (Attention) Effect
2. John Henry Effect
3. Placeo Effect
4 Demand Characteristics

21

What is another name for the Attention Effect?

Hawthorne Effect

22

What happened in the Hawthorne Effect?

Workers productivity went up when lighting was increased AND when it was decreased

23

What is the interpretation of the Hawthorne Effect?

workers respond to researcher attention whether or not the lighting was increased or decreased

24

What controls were used in Hawthorne Confound?

1. Experimental group
2. control receiving attention
3. control NOT receiving attention

25

What is the John Henry Effect?

possibility that the control group might become aware of its 'inferior' status and respond by trying to outperform the experimental group

26

What is the John Henry Effect controlled by?

concealing the control group's knowledge of its status

27

What is the Placebo Effect?

tendency of individuals to improve (or feel improved) simply because they know they are being treated

28

Placebo Effect controlled by...

giving the control group a placebo which provides treatment for the control group

29

What is a Demand Characteristic?

a cue that lets participants know the expected outcomes of an experiment

30

Demand Characteristics are controlled by....

concealing researcher expectations from participants

31

Confounding is a threat to _________ ____________?

Internal Validity

32

What is Internal Validity?

1. the degree to which the relationship between the scores reflects only the relationship between the intended variables
2. Changes in the Y (DV) related directly and only to changes in the X (IV)

33

What is Internal Validity?

1. the degree to which the relationship between the scores reflects only the relationship between the intended variables
2. Changes in the Y (DV) related directly and only to changes in the X (IV)

34

List 7 Common threats to Internal Validity.

1. History
2. Maturation
3. Instrumentation
4. Testing
5. Statistical Regression
6. Selection Bias
7. Mortality

35

What are 2 factors in Historical threats to Internal Validity?

1. environmental influences between pre and post tests
2. non-treatment events occurring between observations

36

Name 2 Factors in Maturational Threats to Internal Validity.

1. participants age between pre and post tests
2. performance changes due to aging and confounds the effect of the treatment

37

Name 2 Factors in Instrumentational threats to Internval Validity.

1. Potential changes in measurement procedures from pretest to post test administration
2. unobserved change in observer criteria or instrument calibration confound the effects of treatment

38

Name 2 Factors in Testing Threats to Internal Validity

1. learning occurs during pretest and affects post test results
2. testing prior to treatment changes how people respond in post-testing

39

What is the statistical regression threat to internal validity?

participants selected for treatment on the basis of their extreme scores tend to move closer to the mean on retesting

40

What is the Selection Bias threat to internal validity?

nonequivalent initial groups used

41

What is the Mortality threat to internal validity?

differential loss of participants from the groups to be compared resulting in nonequivalent groups

42

What is the single best approach to controlling threats to Interval Validity?

Experimental Design that incorporates random assignment of participants to experimental and control groups

43

List FOUR Common threats to External Validity.

1. Interaction of Selection Bias with IV
2. Reactive effects of experimental arrangements
3. Reactive effects of testing (pretest sensitization)
4. Multiple Treatment Interference

44

Describe the Interaction of Selection Bias with IV (threat to External Validity)

1. If a biased sample of participants is used in an experiment, a researcher will not know whether the effects of the treatment can be expected if the treatment is administered to the entire population
2. Results of the study may only pertain to the particular type of participants that were included in the study

45

Interaction of Selection Bias with IV is controlled by.....

random selection

46

Describe the Reactive effects of experimental arrangements (Threat to external validity)

1. If the experimental setting is different from the natural setting in which the population usually operates, the effects that are observed in the experimental setting may not be generalized to the natural setting
2. The presence of observers, instrumentation, or the lab environment that affects the participant but would not occur if the participant had been in a natural setting

47

Reactive effects of experimental arrangements is controlled by....

experiments conducted under natural conditions when possible

48

Describe Reactive effects of Testing (pretest sensitization). (Threat to External Validity)

1. Possibility that the pretest might influence HOW the participants respond to the experimental treatment
2. Pretesting may decrease or increase the sensitivity of the participants to the IV. Thus, one might not be able to generalize to non-pretested individuals.

49

Reactive effects of testing is controlled by....

conducting a true experiment without pretests

50

Describe Multiple Treatment inference in Threats to external Validity.

1. Occurs when a group of participants is given more than one treatment
2. Treatments given earlier in an experiment might affect the effectiveness of treatments given later to the same participants
3. Cannot generalize to single treatment conditions

51

Multiple treatment interference is controlled by....

not using multiple treatments in the experiment

52

Question answered by the meaning of Internal Validity is...

Is the treatment responsible for the observed change?

53

Question answered by the meaning of External Validity is...

To whom can the results be generalized?

54

What happens to internal and external validity when a control is used to reduce the effect of confounding and extraneous variables?

1. Internal Validity increases
2. External Validity decreases

55

What happens to internal and external validity when steps are taken to make the experimental setting as natural as possible?

1. Internal Validity decreases
2. External Validity increases

56

Who are Campbell and Stanley?

developers of symbols used to represent research designs

57

What is the symbol for Random assignment to groups?

R

58

What is the symbol for Observation or measurement?

O

59

What is the symbol for Experimental treatment?

X

60

What is the symbol indicating a control condition?

(blank space)

61

Define a control condition.

1. May contain some kind of treatment.
2. is the standard by which the effectiveness of the experimental treatment is judged

62

What does the symbol X stand for?

Experimental treatment

63

What does the symbol O stand for?

Observation or measurement

64

What does the symbol R stand for?

Random assignment to groups

65

______________ ___________ designs are characterized by poor internal validity.

Pre-experimental

66

Pre-experimental designs are characterized by _________ ________ __________.

Poor internal validity

67

What are Pre-experimental designs used for?

preliminary pilot studies in which researchers are trying out potential treatment and measuring tools to learn about acceptability and accuracy

68

List 3 examples of Pre-experimental designs.

1. One-Group Pretest-Posttest Design
2. One-Shot Case Study
3. Static-Group Comparison Design

69

Define One-Group Pretest-Posttest Design

one group is:
a. pretested
b. treated with IV
c. post-tested

70

Define One-Shot Case Study

One participant is:
a. treated with IV
b. Post-tested

71

Define One-Shot Case Study

One participant is:
a. treated with IV
b. Post-tested

72

Define Static-Group Comparison Design

Group A
a. Treated with IV
b. Post-tested
Group B
a. Post-tested

73

Quasi-experimental Designs are of intermediate value for exploring ___________ and _________ relationships.

cause and effect

74

____________ -__________ designs are of intermediate value for exploring cause and effect relationships.

quasi-experimental

75

Quasi-experimental designs are used when...

random assignment to groups is not possible

76

List 2 examples of Quasi-experimental designs.

1. nonequivalent control group design
2. Equivalent time-samples design

77

Define Nonequivalent Control Group Design

Group A is:
a. Pretested
b. treated with IV
c. Post-tested

Group B is:
a. Pretested
b. Post-tested

78

Strengths of Nonequivalent Control Group Design

although participants are not assigned at random to the two groups, researchers often use some form of matching to increase the internal validity of the results

79

Strength of Nonequivalent Control Group Design

although participants are not assigned at random to the two groups, researchers often use some form of matching to increase the internal validity of the results

80

Define Equivalent Time-Sample Design

ABAB Design
Group/Individual A has:
a. Multiple initial observations before treatment (baseline)
b. Multiple and alternating control and experimental treatments

81

Two Strengths of Equivalent Time-Samples Design (ABAB Design)

1. researchers know that the experimental participants and control participants (who are actually the same individuals) are identical
2. Unlikely that there are threats to internal validity, since it is unlikely that they are operating only at the times when X1 is given and not when Xo is given

82

In the Equivalent Time-Samples Design (ABAB), what is the threat to external validity?

multiple treatment interference

83

True Experimental Designs are characterized by

random assignment to treatments

84

_________ __________ designs are characterized by random assignments to treatments.

true experimental

85

List 3 Examples of True Experimental Designs

1. Pretest-Posttest Randomized Control Group Design
2. Posttest-Only Randomized Control Group Design
3. Solomon Randomized Four-Group Design

86

Define Pretest-Posttest Randomized Control Group Design

Group A is:
a. randomly assigned
b. pretested
c. treated with IV
d. posttested

Group B
a. randomly assigned
b. pretested
c. posttested

87

List of 2 Strengths of Pretest-Posttest Randomized Control Group Design

1. Random assignment
2. Use of pretest

88

By assigning participants at random to groups, researchers are assured that there are no ___________ _____________ in the assignment.

systematic biases

89

Comparison of posttest and pretest results allow researcher to determine ____________ _________ each group gained, not just whether they are ___________ at the end of the experiment.

a. how much
b. different

90

Define Posttest-Only Randomized Control Group Design

Group A has:
a. random assignment
b. treated with IV
c. post tested

Group B has:
a. random assignment
b. post tested

91

Define Posttest-Only Randomized Control Group Design

Group A has:
a. random assignment
b. treated with IV
c. post tested

Group B has:
a. random assignment
b. post tested

92

List 2 Strengths of Posttest-Only Random assignment

1. Random Assignment
2. No Pretest sensitization

93

List 2 positive factors of random assignment

1. assures compatibility of groups as with pretest-posttest control group design
2. assures that there are no biases in assignment as with pretest-posttest randomized control group design

94

Describe Solomon Randomized Four-Group Design

Group 1 is:
a. randomized
b. pretested
c. treated with IV
d. posttested

Group 2 is:
a. randomized
b. pretested
c. posttested

Group 3 is:
a. randomized
b. treated with IV
c posttested

Group 4 is:
a. randomized
b. posttested

95

Major strength of Solomon Randomized Four-Group Design

allows for all comparisons needed

96

What factor is needed to execute a Solomon Randomized Four-Group Design?

reasonably large pool of participants to start with so that a minimum of 30 per group is in the final sample

97

Why is there a minimum of 30 participants per group needed in a Solomon Randomized Four Group Design?

to ensure a sufficient number to perform statistically significant tests

98

List 3 reasons why Experimental Designs are best.

1. Co-variation between X (IV) variable and Y (DV) variable
2. Temporal Sequence, X (IV) variable occurs before Y (DV) variable
3. Control of third (extraneous or confounding) variables

99

What is 'co-variation between X (IV) variable and Y (DV) variable'?

1. researcher manipulates the IV
2. Then measures the results of that manipulation in the DV

100

What is "temporal sequence, X (IV) variable occurs before Y (DV) variable"?

1. researcher presents the conditions of the IV
2. Then measures the results in the DV

101

What is "control of third variables (extraneous or confounding)"?

1. researcher tries to eliminate or control the threats to internal validity
2. threats that cannot be eliminated or controlled are balanced through random assignment of participants to experiment/treatment and control conditions/groups

102

List 3 reasons why TRUE experimental designs are better than Quasi-experimental designs.

1. QE has NOT random assignment to groups or conditions. TE does
2. in QE the IV is not manipulated by researcher and in TE it is
3. in QE there is only some control of third variables and in TE there is

103

What is the definition of Meta-Analysis?

1. statistical methods for combining the results of previous studies on a given topic
2. mathematically synthesizing the statistical results of previous studies on a given topic

104

What does 'meta' mean?

occurring later and/or being later and more highly organized

105

What does 'analysis' mean?

synthesis

106

List 4 components in interpreting the differences found in the results of previous studies.

1. random sampling errors created by assigning participants at random to groups
2. random errors of measurement
3. systematic errors known to one or more of the researchers
4. systematic errors of which one or more of the researchers are unaware

107

List 3 important characteristics of meta-analysis.

1. statistical results are based on the combined sample size of all the studies (BIG samples sizes yield more reliable results)
2. synthesis is of the result of studies conducted by independent researchers
3. process of averaging when conducting a meta-analysis tends to decrease the effects of both the random and systematic errors identified in previous slides

108

List 2 reasons why the synthesis of meta-analysis studies is better.

1. systematic errors of one researcher are unlikely to be committed by other researchers
2. the effects of one researchers systematic errors are moderated when averaged with the results obtained by other independent researchers who have not made the same error

109

What symbols are used to denote measure of effect size?

d
r (squared)

110

List the 3 steps of calculating effect size.

1. calculate d (r squared)
2. average the values of d (r squared)
3. if the studies involve varying number of participants, the averaged value of d (r squared) can be weighted to give more weight in the final average to those studies with more participants

111

List 3 Strengths of Meta-Analysis.

1. produces results based on large combined samples
2. synthesizes studies by independent researchers
3. objectivity of conclusions

112

List 2 potential weaknesses of meta-analysis.

1. risk of poor selection of studies
2. risk of publication bias

113

Because it is not possible to anticipate all potential for harm researchers must _________.

have a research committee review research plans for potential harm

114

List 2 considerations in right to privacy in research.

1. conduct research with concern for the dignity and welfare of participants.
2. psychologist inform research participants of their anticipated sharing of further use of personally identifiable research data and of the possibility of unanticipated future uses

115

List 3 measures to keep collected data confidential.

1. disclose only group level data
2. disclose in statistical form
3 change names

116

If participants are entitled to know the purpose of the research before agreeing to participate, what is the risk?

knowledge of the purpose of the study may bias the results

117

List 5 specifics to be included in the informed consent

1. general purpose of the research
2. what will be done to them during the research
3. what the potential benefit(s) to them and others might be
4. what the potential for harm to them might be
5. the fact that they may withdraw at any time without penalty, even midstream

118

List 5 specifics included in a research participant debriefing

1. Review of purpose(s) of the study and the procedures used
2. offer to share the results with the participants when the results become available
3. Reassurance that the data will remain confidential
4. Opportunity for participants to ask for information about any aspect of the study in which they participated
5. Information about how to contact the researcher in the future, either for more information or to request assistance with any harmful effects that might reveal themselves at a later data

119

What is the one caution to be aware of during debriefing?

researchers should try to identify participants who may need more help in overcoming unanticipated harm to them than a standard debriefing session provides

120

List 3 actions to be done for a participant after research.

1. provide them with information about how to contact the researcher in the future
2. offer assistance
3. request an additional follow-up visit