Ch. 5

Question 1

experiment

Answer 1

A type of study designed specifically to answer the question of whether there is a causal relationship between two variables.

Question 2

independent variable

Answer 2

The variable the experimenter manipulates.

Question 3

dependent variable

Answer 3

The variable the experimenter measures (it is the presumed effect).

Question 4

conditions

Answer 4

The different levels of the independent variable to which participants are assigned.

Question 5

control

Answer 5

Holding extraneous variables constant in order to separate the effect of the independent variable from the effect of the extraneous variables.

Question 6

extraneous variables

Answer 6

Any variable other than the dependent and independent variable.

Question 7

manipulate

Answer 7

Changing the level, or condition, of the independent variable systematically so that different groups of participants are exposed to different levels of that variable, or the same group of participants is exposed to different levels at different times.

Notice that the manipulation of an independent variable must involve the active intervention of the researcher.

Comparing groups of people who differ on the independent variable before the study begins is not the same as manipulating that variable.

there are many situations in which the independent variable cannot be manipulated for practical or ethical reasons and therefore an experiment is not possible.

Question 8

single factor two-level design

Answer 8

An experiment design involving a single independent variable with two conditions.

Question 9

single factor multi level design

Answer 9

When an experiment has one independent variable that is manipulated to produce more than two conditions.

Question 10

Control of Extraneous Variables

Answer 10

Extraneous variables pose a problem because many of them are likely to have some effect on the dependent variable.

This influencing factor can make it difficult to separate the effect of the independent variable from the effects of the extraneous variables, which is why it is important to control extraneous variables by holding them constant.

Question 11

Extraneous Variables as “Noise”

Answer 11

Extraneous variables make it difficult to detect the effect of the independent variable in two ways.

One is by adding variability or “noise” to the data.

Thus one reason researchers try to control extraneous variables is so their data look more like the idealized data in Table 5.1, which makes the effect of the independent variable easier to detect (although real data never look quite that good).

One way to control extraneous variables is to hold them constant.

This technique can mean holding situation or task variables constant by testing all participants in the same location, giving them identical instructions, treating them in the same way, and so on.

It can also mean holding participant variables constant.

The obvious downside to this approach is that it would lower the external validity of the study—in particular, the extent to which the results can be generalized beyond the people actually studied.

Question 12

confounding variable

Answer 12

An extraneous variable that varies systematically with the independent variable, and thus confuses the effect of the independent variable with the effect of the extraneous one.

To confound means to confuse, and this effect is exactly why confounding variables are undesirable.

Because they differ systematically across conditions—just like the independent variable—they provide an alternative explanation for any observed difference in the dependent variable.

One way to avoid confounding variables is by holding extraneous variables constant.

Question 13

treatment

Answer 13

Any intervention meant to change people’s behavior for the better.

This intervention includes psychotherapies and medical treatments for psychological disorders but also interventions designed to improve learning, promote conservation, reduce prejudice, and so on.

To determine whether a treatment works, participants are randomly assigned to either a treatment condition or control condition

If participants in the treatment condition end up better off than participants in the control condition then the researcher can conclude that the treatment works.

Question 14

treatment condition

Answer 14

The condition in which participants receive the treatment.

Question 15

control condition

Answer 15

The condition in which participants do not receive the treatment.

Question 16

randomized clinical trial

Answer 16

An experiment that researches the effectiveness of psychotherapies and medical treatments.

Question 17

no-treatment control condition

Answer 17

The condition in which participants receive no treatment whatsoever.

Question 18

placebo

Answer 18

A simulated treatment that lacks any active ingredient or element that is hypothesized to make the treatment effective, but is otherwise identical to the treatment.

Question 19

placebo effect

Answer 19

An effect that is due to the placebo rather than the treatment.

they are probably driven primarily by people’s expectations that they will improve.

Having the expectation to improve can result in reduced stress, anxiety, and depression, which can alter perceptions and even improve immune system functioning.

pose a serious problem for researchers who want to determine whether a treatment works.

Question 20

placebo control condition

Answer 20

Condition in which the participants receive a placebo rather than the treatment.

placebo that looks much like the treatment but lacks the active ingredient or element thought to be responsible for the treatment’s effectiveness.

Question 21

wait-list control condition

Answer 21

Condition in which participants are told that they will receive the treatment but must wait until the participants in the treatment condition have already received it.

Of course, the principle of informed consent requires that participants be told that they will be assigned to either a treatment or a placebo control condition—even though they cannot be told which until the experiment ends.

In many cases the participants who had been in the control condition are then offered an opportunity to have the real treatment.

This disclosure allows researchers to compare participants who have received the treatment with participants who are not currently receiving it but who still expect to improve (eventually).

Question 22

A final solution to the problem of placebo effects

Answer 22

is to leave out the control condition completely and compare any new treatment with the best available alternative treatment.

Question 23

between-subjects experiment

Answer 23

An experiment in which each participant is tested in only one condition.

This matching is a matter of controlling these extraneous participant variables across conditions so that they do not become confounding variables.

Question 24

random assignment

Answer 24

Means using a random process to decide which participants are tested in which conditions.

Do not confuse random assignment with random sampling.

Random sampling is a method for selecting a sample from a population, and it is rarely used in psychological research.

Random assignment is a method for assigning participants in a sample to the different conditions,

Question 25

random assignment should meet two criteria.

Answer 25

One is that each participant has an equal chance of being assigned to each condition (e.g., a 50% chance of being assigned to each of two conditions). The second is that each participant is assigned to a condition independently of other participants. Thus one way to assign participants to two conditions would be to flip a coin for each one. If the coin lands heads, the participant is assigned to Condition A, and if it lands tails, the participant is assigned to Condition B. For three conditions, one could use a computer to generate a random integer from 1 to 3 for each participant.

Question 26

One problem with coin flipping and other strict procedures for random assignment is

Answer 26

that they are likely to result in unequal sample sizes in the different conditions. Unequal sample sizes are generally not a serious problem, and you should never throw away data you have already collected to achieve equal sample sizes. However, for a fixed number of participants, it is statistically most efficient to divide them into equal-sized groups. It is standard practice, therefore, to use a kind of modified random assignment that keeps the number of participants in each group as similar as possible.

Question 27

block randomization

Answer 27

All the conditions occur once in the sequence before any of them is repeated. Within each of these “blocks,” the conditions occur in a random order. Again, the sequence of conditions is usually generated before any participants are tested, and each new participant is assigned to the next condition in the sequence. Random assignment is not guaranteed to control all extraneous variables across conditions. The process is random, so it is always possible that just by chance, the participants in one condition might turn out to be substantially older, less tired, more motivated, or less depressed on average than the participants in another condition. However, there are some reasons that this possibility is not a major concern. One is that random assignment works better than one might expect, especially for large samples. Another is that the inferential statistics that researchers use to decide whether a difference between groups reflects a difference in the population takes the “fallibility” of random assignment into account. Even if random assignment does result in a confounding variable and therefore produces misleading results, this confound is likely to be detected when the experiment is replicated.

Question 28

Matched-Groups design

Answer 28

An experiment design in which the participants in the various conditions are matched on the dependent variable or on some extraneous variable(s) prior the manipulation of the independent variable. This guarantees that these variables will not be confounded across the experimental conditions.

Question 29

Within-Subjects Experiments

Answer 29

An experiment in which each participant is tested under all conditions. The primary advantage of this approach is that it provides maximum control of extraneous participant variables. Within-subjects experiments also make it possible to use statistical procedures that remove the effect of these extraneous participant variables on the dependent variable and therefore make the data less “noisy” and the effect of the independent variable easier to detect. Within-subjects experiments also make it easier for participants to guess the hypothesis

Question 30

order effect

Answer 30

An effect that occurs when participants' responses in the various conditions are affected by the order of conditions to which they were exposed.

Question 31

carryover effect

Answer 31

An effect of being tested in one condition on participants’ behavior in later conditions.

Question 32

practice effect

Answer 32

An effect where participants perform a task better in later conditions because they have had a chance to practice it.

Question 33

fatigue effect

Answer 33

An effect where participants perform a task worse in later conditions because they become tired or bored.

Question 34

context effect (or contrast effect)

Answer 34

Unintended influences on respondents’ answers because they are not related to the content of the item but to the context in which the item appears.

Question 35

counterbalancing

Answer 35

Varying the order of the conditions in which participants are tested, to help solve the problem of order effects in within-subjects experiments.

Question 36

complete counterbalancing

Answer 36

A method in which an equal number of participants complete each possible order of conditions.

Question 37

Latin square design

Answer 37

A more efficient way of counterbalancing is through a Latin square design which randomizes through having equal rows and columns.

Question 38

random counterbalancing

Answer 38

A method in which the order of the conditions is randomly determined for each participant. Using this technique every possible order of conditions is determined and then one of these orders is randomly selected for each participant. This is not as powerful a technique as complete counterbalancing or partial counterbalancing using a Latin squares design. Use of random counterbalancing will result in more random error, but if order effects are likely to be small and the number of conditions is large, this is an option available to researchers.

Question 39

There are two ways to think about what counterbalancing accomplishes

Answer 39

One is that it controls the order of conditions so that it is no longer a confounding variable. Thus any overall difference in the dependent variable between the two conditions cannot have been caused by the order of conditions. A second way to think about what counterbalancing accomplishes is that if there are carryover effects, it makes it possible to detect them. One can analyze the data separately for each order to see whether it had an effect.

Question 40

Simultaneous Within-Subjects Designs

Answer 40

often used when participants make multiple responses in each condition. Imagine, for example, that participants judge the guilt of 10 attractive defendants and 10 unattractive defendants. Instead of having people make judgments about all 10 defendants of one type followed by all 10 defendants of the other type, the researcher could present all 20 defendants in a sequence that mixed the two types. The researcher could then compute each participant’s mean rating for each type of defendant.

Question 41

Between-Subjects or Within-Subjects?

Answer 41

Between-subjects experiments have the advantage of being conceptually simpler and requiring less testing time per participant. They also avoid carryover effects without the need for counterbalancing. Within-subjects experiments have the advantage of controlling extraneous participant variables, which generally reduces noise in the data and makes it easier to detect any effect of the independent variable upon the dependent variable. Within-subjects experiments also require fewer participants than between-subjects experiments to detect an effect of the same size. A good rule of thumb, then, is that if it is possible to conduct a within-subjects experiment (with proper counterbalancing) in the time that is available per participant—and you have no serious concerns about carryover effects—this design is probably the best option. If a within-subjects design would be difficult or impossible to carry out, then you should consider a between-subjects design instead. Remember also that using one type of design does not preclude using the other type in a different study. There is no reason that a researcher could not use both a between-subjects design and a within-subjects design to answer the same research question. In fact, professional researchers often take exactly this type of mixed methods approach

Question 42

Four Big Validities

Answer 42

internal validity, external validity, construct validity, and statistical validity.

Question 43

Internal Validity

Answer 43

Refers to the degree to which we can confidently infer a causal relationship between variables. Thus experiments are high in internal validity because the way they are conducted—with the manipulation of the independent variable and the control of extraneous variables (such as through the use of random assignment to minimize confounds)—provides strong support for causal conclusions. In contrast, non-experimental research designs (e.g., correlational designs), in which variables are measured but are not manipulated by an experimenter, are low in internal validity. The purpose of an experiment, however, is to show that two variables are statistically related and to do so in a way that supports the conclusion that the independent variable caused any observed differences in the dependent variable. The logic is based on this assumption: If the researcher creates two or more highly similar conditions and then manipulates the independent variable to produce just one difference between them, then any later difference between the conditions must have been caused by the independent variable.

Question 44

External Validity

Answer 44

Refers to the degree to which we can generalize the findings to other circumstances or settings, like the real-world environment. As a general rule, studies are higher in external validity when the participants and the situation studied are similar to those that the researchers want to generalize to and participants encounter every day, often described as mundane realism

Question 45

psychological realism

Answer 45

Where the same mental process is used in both the laboratory and in the real world.

Question 46

mundane realism

Answer 46

When the participants and the situation studied are similar to those that the researchers want to generalize to and participants encounter every day.

Question 47

should be careful, however, not to draw the blanket conclusion that experiments are low in external validity

Answer 47

One reason is that experiments need not seem artificial. A second reason not to draw the blanket conclusion that experiments are low in external validity is that they are often conducted to learn about psychological processes that are likely to operate in a variety of people and situations.

Question 48

Construct Validity

Answer 48

One of the "big four" validities, whereby the research question is clearly operationalized by the study's methods. the quality of the experiment’s manipulations. very high because the experiment’s manipulations very clearly speak to the research question.

Question 49

operationalization

Answer 49

The specification of exactly how the research question will be studied in the experiment design.

Question 50

Statistical Validity

Answer 50

Concerns the proper statistical treatment of data and the soundness of the researchers’ statistical conclusions. When considering the proper type of test, researchers must consider the scale of measure their dependent variable was measured on and the design of their study. Further, many inferential statistics tests carry certain assumptions (e.g., the data are normally distributed) and statistical validity is threatened when these assumptions are not met but the statistics are used nonetheless. _{One common critique of experiments is that a study did not have enough participants. The main reason for this criticism is that it is difficult to generalize about a population from a small sample. At the outset, it seems as though this critique is about external validity but there are studies where small sample sizes are not a problem. small sample sizes are actually a critique of statistical validity. The statistical validity speaks to whether the statistics conducted in the study are sound and support the conclusions that are made.} the likelihood of detecting an effect of the independent variable on the dependent variable depends on not just whether a relationship really exists between these variables, but also the number of conditions and the size of the sample.

Question 51

Prioritizing Validities

Answer 51

These four big validities–internal, external, construct, and statistical–are useful to keep in mind when both reading about other experiments and designing your own. However, researchers must prioritize and often it is not possible to have high validity in all four areas. This discrepancy does not invalidate the study but it shows where there may be room for improvement for future follow-up studies. Morling (2014) points out that many psychology studies have high internal and construct validity but sometimes sacrifice external validity

Question 52

Practical Considerations: Recruiting Participants- subject pool

Answer 52

An established group of people who have agreed to be contacted about participating in research studies.

Question 53

Practical Considerations: Recruiting Participants- The Volunteer Subject

Answer 53

Even if the participants in a study receive compensation in the form of course credit, a small amount of money, or a chance at being treated for a psychological problem, they are still essentially volunteers. This is worth considering because people who volunteer to participate in psychological research have been shown to differ in predictable ways from those who do not volunteer. Specifically, there is good evidence that on average, volunteers have the following characteristics compared with non-volunteers: • They are more interested in the topic of the research. • They are more educated. • They have a greater need for approval. • They have higher IQ. • They are more sociable. • They are higher in social class. This difference can be an issue of external validity if there is a reason to believe that participants with these characteristics are likely to behave differently than the general population.

Question 54

Practical Considerations: Recruiting Participants- Advertisements or personal appeals

Answer 54

Participants who are not in subject pools can also be recruited by posting or publishing advertisements or making personal appeals to groups that represent the population of interest.

Question 55

Practical Considerations: Recruiting Participants- In field experiments

Answer 55

In many field experiments, the task is not recruiting participants but selecting them. For example, conducted a field experiment on the effect of being smiled at on helping, in which the participants were shoppers at a supermarket. A confederate walking down a stairway gazed directly at a shopper walking up the stairway and either smiled or did not smile. Shortly afterward, the shopper encountered another confederate, who dropped some computer diskettes on the ground. The dependent variable was whether or not the shopper stopped to help pick up the diskettes. Two aspects of this study that are worth addressing here. First, notice that these participants were not “recruited,” which means that the IRB would have taken care to ensure that dispensing with informed consent in this case was acceptable (e.g., the situation would not have been expected to cause any harm and the study was conducted in the context of people’s ordinary activities). Second, even though informed consent was not necessary, the researchers still had to select participants from among all the shoppers taking the stairs that day. It is extremely important that this kind of selection be done according to a well-defined set of rules that are established before the data collection begins and can be explained clearly afterward. In this case, with each trip down the stairs, the confederate was instructed to gaze at the first person he encountered who appeared to be between the ages of 20 and 50. Only if the person gazed back did they become a participant in the study. The point of having a well-defined selection rule is to avoid bias in the selection of participants.

Question 56

Practical Considerations: Standardizing the Procedure- experimenter expectancy effect

Answer 56

When the experimenter’s expectations about how participants “should” behave in the experiment affect how the participants behave.

Question 57

Practical Considerations: Standardizing the Procedure- Here are several ways to do this

Answer 57

The way to minimize unintended variation in the procedure is to standardize it as much as possible so that it is carried out in the same way for all participants regardless of the condition they are in. • Create a written protocol that specifies everything that the experimenters are to do and say from the time they greet participants to the time they dismiss them. • Create standard instructions that participants read themselves or that are read to them word for word by the experimenter. • Automate the rest of the procedure as much as possible by using software packages for this purpose or even simple computer slide shows. • Anticipate participants’ questions and either raise and answer them in the instructions or develop standard answers for them. • Train multiple experimenters on the protocol together and have them practice on each other. • Be sure that each experimenter tests participants in all conditions.

Question 58

Practical Considerations: Standardizing the Procedure- double-blind study

Answer 58

A method to reduce experimenter bias, where neither the participant nor the experimenter is knowledgeable about the condition to which the participant is assigned.

Question 59

Practical Considerations: Record Keeping-

Answer 59

It is essential to keep good records when you conduct an experiment. It is typical for experimenters to generate a written sequence of conditions before the study begins and then to test each new participant in the next condition in the sequence. As you test them, it is a good idea to add to this list basic demographic information; the date, time, and place of testing; and the name of the experimenter who did the testing. It is also a good idea to have a place for the experimenter to write down comments about unusual occurrences (e.g., a confused or uncooperative participant) or questions that come up. This kind of information can be useful later if you decide to analyze sex differences or effects of different experimenters, or if a question arises about a particular participant or testing session. Since participants’ identities should be kept as confidential (or anonymous) as possible, their names and other identifying information should not be included with their data. In order to identify individual participants, it can, therefore, be useful to assign an identification number to each participant as you test them. Simply numbering them consecutively beginning with 1 is usually sufficient. This number can then also be written on any response sheets or questionnaires that participants generate, making it easier to keep them together.

Question 60

Practical Considerations: Manipulation Check-

Answer 60

Verifying the experimental manipulation worked by using a different measure of the construct the researcher is trying to manipulate. The purpose of a manipulation check is to confirm that the independent variable was, in fact, successfully manipulated. Are particularly important when the results of an experiment turn out null. In cases where the results show no significant effect of the manipulation of the independent variable on the dependent variable, a manipulation check can help the experimenter determine whether the null result is due to a real absence of an effect of the independent variable on the dependent variable or if it is due to a problem with the manipulation of the independent variable. Usually done at the end of the procedure to be sure that the effect of the manipulation lasted throughout the entire procedure and to avoid calling unnecessary attention to the manipulation (to avoid a demand characteristic). However, researchers are wise to include a manipulation check in a pilot test of their experiment so that they avoid spending a lot of time and resources on an experiment that is doomed to fail and instead spend that time and energy finding a better manipulation of the independent variable.

Question 61

Practical Considerations: Pilot Testing-

Answer 61

Is a small-scale study conducted to make sure that a new procedure works as planned. In a pilot test, you can recruit participants formally (e.g., from an established participant pool) or you can recruit them informally from among family, friends, classmates, and so on. The number of participants can be small, but it should be enough to give you confidence that your procedure works as planned. There are several important questions that you can answer by conducting a pilot test: • Do participants understand the instructions? • What kind of misunderstandings do participants have, what kind of mistakes do they make, and what kind of questions do they ask? • Do participants become bored or frustrated? • Is an indirect manipulation effective? (You will need to include a manipulation check.) • Can participants guess the research question or hypothesis (are there demand characteristics)? • How long does the procedure take? • Are computer programs or other automated procedures working properly? • Are data being recorded correctly? To answer some of these questions you will need to observe participants carefully during the procedure and talk with them about it afterward. Participants are often hesitant to criticize a study in front of the researcher, so be sure they understand that their participation is part of a pilot test and you are genuinely interested in feedback that will help you improve the procedure. If the procedure works as planned, then you can proceed with the actual study. If there are problems to be solved, you can solve them, pilot test the new procedure, and continue with this process until you are ready to proceed.