Unit 3: Non-Experimental Research Methods

Question 1

Non-Experimental Research

Answer 1

Research that LACKS:

1. Manipulation of an independent variable

AND/OR

2. Random assignment of participants to conditions, or orders of conditions

Does not identify causal relationship between dependent and independent variables

Question 2

What kinds of research questions are suited to nonexperimental research?

Answer 2

1. Ask about a single variable (rather than relationship between 2 or more variables)
2. Ask about a noncausal relationship between variables (correlation)
3. Ask about a causal relationship, provided the independent variable is not manipulated and participants are not randomly assigned to conditions
4. Ask about a broad or exploratory observation

Question 3

Main types of nonexperimental research

Answer 3

1. Single-variable research
2. Correlational research
3. Quasi-experimental research
4. Qualitative research

Question 4

Single-variable research

Answer 4

Main interest is in one variable

Ex: Milgram’s first obedience study, all participants performed same task under same conditions

Question 5

Correlational research

Answer 5

2 variables of interest, no control of other possible variables

How does change in one variable relate to change in another?

Question 6

Qualitative research

Answer 6

Data are nonnumerical, often categorical

Data cannot be analyzed statistically

Question 7

Strong correlation

Answer 7

Represented by an r value between +/- .7 and .9

x and y values almost always go together; few exceptions

In a scatterplot, data points are more tightly arranged along the sloped line

Question 8

Weak correlation

Answer 8

Represented by an r value between +/- .1 and .3

x and y values go together some of the time; many exceptions

In a scatterplot, data points are very spread out along the sloped line

Question 9

Moderate correlation

Answer 9

Represented by an r value between +/- .4 and .6

Question 10

Types of data collection associated with correlational research

Answer 10

1. Naturalistic observation

2. Archival data

Question 11

Naturalistic observation

Answer 11

Observing behaviour in the environmental setting in which it normally takes place

Participants are usually unaware they are being studied

Researches must consider:

1. Sampling; decide who to observe, what to include in data collection
2. Measurement; decide what specific behaviours to measure, use coding to clearly define a set of target behaviours so they can be categorized, especially when relying on observer’s judgment

Question 12

Archival data

Answer 12

Data that have already been collected for some other purpose

If data cannot be easily quantified, content analysis is used to identify specific keywords, phrases, ideas so that instances can be counted, timed, or analyzed

Question 13

Advantages of Naturalistic Observation

Answer 13

1. Creation of theories, ideas for future research
2. Results are more easily generalized to the real world
3. Allows for study of phenomena that can’t be replicated in an experiment

Question 14

Limitations of Naturalistic Observation

Answer 14

1. Can’t determine causation
2. Prone to bias; researchers’ expectations can impact how they interpret observations
3. Presence of researchers can impact behaviour being observed

Question 15

Advantages of Archival Research

Answer 15

1. Response bias is minimized; researchers are not present when subjects respond
2. Plentiful data already exists; is often cheaper than other research
3. Can be used to confirm if theoretical explanations apply to the real world
4. Help generate new hypotheses to be tested experimentally

Question 16

Limitations of Archival Research

Answer 16

1. Selective deposit; only certain ideas/responses are chosen to be recorded; can be skewed to look more politically correct or socially acceptable, rather than accurately reflect the real world
2. Survival of archives is selective, could reflect bias
3. Data could include errors
4. Definitions of terms change over time; might refer to a different group now than it did 50 years ago
5. Collection of data could be biased; recording a suicide as an accidental death to protect privacy of family
6. Can’t determine causation

Question 17

Case Study

Answer 17

An example of qualitative research

Intensive, thorough examination of one subject (or a small group of subjects) with a unique characteristic

The goal is to understand in detail the individual experience of participant (rather than drawing generalized conclusions about humanity)

Question 18

Advantages of Case Studies

Answer 18

1. Allow study of rare phenomena that may otherwise be unethical to study experimentally
2. Can generate new theories to be tested experimentally
3. Can challenge existing theories; provide counter-examples
4. Can provide tentative support for current theories

Question 19

Limitations of Case Studies

Answer 19

1. Can’t determine causation; confounding variables
2. Interpretation bias; often researcher records and participates in treatment
3. Can’t generalize from a single case
4. Can offer people false hope; one success story can persuade others to choose unproven treatment

Question 20

Internal validity

Answer 20

The extent to which a research design supports the conclusion that manipulation of the independent variable causes a change in the dependent variable; how well the results can determine causation

Experimental research has highest degree

Quasi-experimental research has moderate degree

Non-experimental research has lowest degree

Question 21

External validity

Answer 21

The degree to which a study ensures that potential findings apply to different settings or samples; how generalizable the results are

Relates to how a study was conducted

Are higher when the participants and the situation studied are similar to those that the researchers want to generalize to and participants encounter everyday

Often described as mundane realism

Question 22

Subject variables

Answer 22

Characteristics that participants possess prior to being assigned to a group

Ex: height, weight, sex

Not a true independent variable because researchers do not manipulate them

Question 23

Quasi-Experimental research

Answer 23

Research method resembles an experiment, but the researchers do not directly manipulate the variables

Participants are not randomly assigned to conditions

The independent variable is indirectly manipulated before the dependent variable is measured

Ex: Studies that group males in one condition and females in another condition

Often used when manipulation of variables is impractical, unethical, or impossible

Ex: Used to evaluate the effectiveness of a treatment in a field setting, such as psychotherapy or an educational intervention

Question 24

Advantages of quasi-experimental research

Answer 24

1. Allows study of variables that are impractical, unethical, or impossible to manipulate
2. Easier to set up than standard experiments because there is no need for random assignment of participants
3. High potential for external validity because natural environments are less artificial than a controlled lab setting
4. Natural setting allows for generalizations to be made to larger population

Question 25

Limitations of quasi-experimental research

Answer 25

1. Due to lack of random assignment, it is exposed to confounding variables. Therefore, internal validity is lower than experimental research 2. Can't determine causation because researchers lack total control over extraneous variables

Question 26

Nonequivalent groups design

Answer 26

A between-subjects research design in which participants have not been randomly assigned to conditions A form of quasi-experimental research

Question 27

Pretest-posttest design

Answer 27

When the dependent variable is tested once before treatment is implemented, and once after

Question 28

Regression to the mean

Answer 28

When an individual scores extremely on a variable one time, the next time they will tend to score less extremely (closer to the individual's mean score)

Question 29

Cross-sectional research

Answer 29

Assesses several different groups at approximately the same time and then compares these groups Often used when investigating the relationship between age and other variables of interest

Question 30

Advantages of cross-sectional research

Answer 30

1. Efficient to conduct because many groups can be studied at once 2. Can suggest new relationships between variables and generate new research questions 3. Can support/disconfirm theories studied in artificial settings 4. Allow us to study variables, like age, that cannot be manipulated experimentally

Question 31

Limitations of cross-sectional research

Answer 31

1. They are always confounded, cause‐and‐effect relationships can never be determined 2. The possibility that the groups differ in multiple ways may artificially exaggerate the differences between groups

Question 32

Longitudinal research

Answer 32

Involve following a group of individuals over time and repeatedly measuring their behaviour Often used in developmental psychology when the subject‐variable age is studied

Question 33

Selective attrition

Answer 33

In longitudinal research, the non‐random loss of participants over time Ex: Sprinters that remain competitive are more likely to continue participating than sprinters who have become slower and stopped competing

Question 34

Advantages of longitudinal research

Answer 34

1. Individual differences are minimized because the same participants are measured over time 2. Can suggest new relationships between variables and generate new research questions 3. Can support/disconfirm theories studied in artificial settings 4. Allow us to study variables, like age, that cannot be manipulated experimentally

Question 35

Limitations of longitudinal research

Answer 35

1. Relatively expensive in terms of time, money, and effort; difficult to track down participants after years have passed 2. Selective attrition can bias samples 3. They are always confounded, cause‐and‐effect relationships can never be determined 4. Differences in time between measurements may artificially minimize any differences between times 5. Vulnerable to practice effect

Question 36

Survey research

Answer 36

Consists of administering a series of predetermined questions to a group of individuals When every individual in a population is included, it is called a census When data is collected from a subset of the population, it is called a sample

Question 37

Advantages of survey research

Answer 37

1. Can be done quickly 2. Large samples can be done with a low cost 3. Anonymity allows study of personal or embarrassing subjects 4. Online surveys can be configured to counterbalance the order of questions, or even to tailor subsequent questions based on the respondent's earlier responses 5. Can suggest new relationships between variables and generate new research questions

Question 38

Limitations of survey research

Answer 38

1. Respondents may not understand the questions 2. Order questions are answered may influence responses; not way to guarantee it is completed in order 3. Results are only as good as the sample that completes the survey; will include any sampling bias 4. Responses can be influenced by "demand characteristics"; participants interpret the researchers' purpose and change responses 5. The format of the question may also influence responding

Question 39

Item-order effect

Answer 39

When the order in which survey items are presented affects people’s responses

Question 40

Common threats to internal validity (in nonexperimental design)

Answer 40

Acronym "MRS SMITH": 1. Maturation 2. Regression to the mean 3. Selection of subjects 4. Selection of maturation interaction 5. Mortality 6. Instrumentation 7. Testing 8. History

Question 41

Maturation (in relation to internal validity)

Answer 41

Physiological processes that occur in the participant that could explain a change in behaviour Changes in scores between testing sessions may simply be due to the passage of time rather than any treatment effects Ex: natural aging; hunger; fatigue

Question 42

Regression to the mean (in relation to internal validity)

Answer 42

The tendency that participants who receive extreme scores when tested, tend to have less extreme scores on subsequent retesting even in the absence of any treatment effects

Question 43

Selection of subjects (in relation to internal validity)

Answer 43

Any bias in selecting and assigning participants to groups that results in systematic differences between the participants in each group.

Question 44

Selection of maturation interaction (in relation to internal validity)

Answer 44

The treatment and no-treatment groups, although similar at one point, would have grown apart (developed differently) even if no treatment had been administered

Question 45

Mortality (in relation to internal validity)

Answer 45

Differential dropping out of some subjects from the comparison groups before the experiment is finished, resulting in differences between the groups that may be unrelated to the treatment effects

Question 46

Instrumentation (in relation to internal validity)

Answer 46

Changes in the measurement procedures may result in differences between the comparison groups that are confused with the treatment effects

Question 47

Testing (in relation to internal validity)

Answer 47

When participants are repeatedly tested, changes in test scores may be more due to practice or knowledge about the test procedure gained from earlier experiences rather than any treatment effects

Question 48

History (in relation to internal validity)

Answer 48

Extraneous events occurring during the course of the experiment that may affect the participants’ responses on the dependent measure Ex: Major social events, equipment malfunctions within experimental situation