Test 2 Flashcards Preview

SOC. 403 > Test 2 > Flashcards

Flashcards in Test 2 Deck (123)
Loading flashcards...
1

What is conceptualization?

-Mental work which consists in indication precisely the MEANINGS of our concepts

-First step in the process of transforming concepts into variables and attributes

-The concept summarizes our conceptions (mental images) of a phenomenon

-Ex: Dave is depressed → come up with mental images/concepts of what that is -- concept summaries

2

How do we conceptualize?

-In order to translate a proposition into a hypothesis, you must first translate the concepts which make up the proposition into the variables which make up the hypothesis
-A proposition is made up of concepts
-A hypothesis is made of variables

-1. Providing a working definition of the concept

-2. Developing the various dimensions of the concept

-3. Providing the indicators of the concept

3

When do we conceptualize?

-Deductive Path: Theory → Axioms → Concepts → Variables → Observations
-Theory → Laws → Props (concepts) → Hypothesis (variables) → Observations

-Conceptualize between “concepts” and “variables” (between 3rd and 4th steps)

-Conceptualize between propositions and hypothesis

-Conceptualization is the first step in the process of transforming concepts into variables and attributes

4

What are the different steps of conceptualization?

-1. Provide
-a. nominal/working definition of the concept
-First define the concept with other words, and explain how we will use it in the research
-Ex: depression is a psychological condition in which a person experiences deep, unshakable sadness and diminished interest in nearly all activities
-Ex: Social class refers to a group of people who share similar life conditions, ways of life, attitudes, beliefs, and access to valued social resources (typically measured by collecting data on income, education, profession)
-b. Its dimensions
-Ex: Depression is characterized by 5 or more of the following or a period of 2 weeks (really long definition that includes affective, cognitive, and physiological dimensions)
-Ex: Dimensions of social class = income, education, profession

-2. Provide operational definition (indicators of the dimensions)
-Concrete manifestations of the various dimensions
-”Symptoms” or “signs” of the various dimensions
-Ex: For affective dimension = worthlessness, guilt, sadness; for cognitive = remembering, focusing, deciding, suicidal thoughts; for physiological = appetite, sleep, energy, fatigue
-Ex: For income = salary, mortgage; for education = highest diploma, years of schooling; for profession = title, prestige score

5

What is operationalization?

-Decisions about how we will measure/observe the indicators of the various dimensions

-Operationalization is the second step in the process of transforming concepts into variables and attributes

-In some cases, operationalization entails asking respondents to answer questions

-These questions are the variables

-The answers to these questions are the attributes of these variables

-Refers to decisions the researcher makes about how to collect observations

6

Why do we conceptualize? How do we conceptualize? When do we conceptualize?

-WHY: Because it’s necessary to clarify what we mean by these concepts in order to draw meaningful conclusions about them

-HOW: Sometimes by asking questions

WHEN: Second step in process of transforming concepts into variables and attributes; after conceptualization; between concepts and variables; after we have a well-defined purpose and a clear description of the kinds of outcomes you want to achieve

7

Types of operationalization?

-Open-Ended Questions
-Respondent provides his/her answers to our questions
-Ex: In your opinion, what are the main crises facing America today?

-Closed-Ended Questions
-Respondents must choose from given answers
-Ex: Which of the following is the main crisis facing America today? Hunger, Global Warming, Terrorism, or AIDS

8

Advantages and Disadvantages of open-ended questions?

-Advantages:
-Allow the discovery of information
-Allow respondents to choose their answers

-Disadvantages:
-May be difficult to organize and code
-Often skipped in a questionnaire

9

Advantages and Disadvantages of closed-ended questions?

-Advantages:
-Allow the standardization of answers
-Enable easy organization and coding

-Disadvantages:
-Force respondents to choose answers
-Limit information

10

Levels of Questions/Attributes?

-1. Nominal

-2. Ordinal

-3. Interval

-4. Ratio

-More specifically, the 4 levels of qualities of closed-ended questions

11

Nominal

-A nominal question/variable measures attributes that are:
-1. Exhaustive
-Attributes must include all possibilities
-Every respondent should be able to find his/her appropriate attribute

-2. Mutually exclusive:
-Respondents should be able to choose only one attribute

-3. Having no inherent order:
-No answer is greater than another
-Ex: Gender, religion, country of birth, marital status, etc.
-Attributes that are generally good or generally bad

-Indicator: Difficulty Focusing
-Nominal Variable: Lately, I seem to have a difficult time focusing on the task at hand. YES or NO.
-IT IS: exhaustive, mutually exclusive, no order

-Want to operationalize “Religion”
-Indicator: Religion
-Question or Nominal Variable: What is your religion? (Check one)
-Buddhist
-Protestant
-Catholic
-Jewish
-Muslim
-Christian
-Mormon

-Problem with question above?
-Not exhaustive (some religions missing)
-Not quite mutually exclusive (should I pick “Christian” or “Catholic”)

-Indicator: Main Data Source in Research
-Nominal Variable: Which of the following data sources do you mostly use in your research?
-TV
-Movies
-Newspapers
-Material Culture
-Personal Diaries
-Mass Media
-Historical Documents
-Legal Records

-Problem with question above?
-Not exhaustive
-No mutually exclusive

12

Ordinal

-An ordinal variable measures attributes that are:
-1. Mutually exclusive
-2. Exhaustive
-3. Ranked according to some logical order

-Indicator: Highest Diploma
-Ordinal Variable: What is the highest degree or diploma you have obtained?
-Some high school
-High school diploma
-Some college
-BA/BS
-MA/MS
-PhD
-*Lower to Higher

-Above question is exhaustive, mutually exclusive, logical order

-Indicator: Social Class
-Ordinal Variable: Which social class do you belong to?
-Upper-Class
-Upper-Middle Class
-Middle Class
-Lower Middle Class
-Working Class
-Lower Class
-Don’t Know
-*Higher to Lower

-Above question is exhaustive, mutually exclusive, logical order’’

-Indicator: Concentrating
-Ordinal Variable: Sometimes, I find it hard to concentrate on a particular task for long periods of time.
-Strongly Agree
-Agree
-Don’t Know
-Disagree
-Strongly Disagree

-Problems: Don’t know the “space” people put between different answers (like between strongly agree and agree)

13

Interval

-An interval variable measures attributes that are:
-1. Mutually exclusive
-2. Exhaustive
-3. Ranked according to some logical order
-4. Assigned a standardized numerical value
-IQ, MMPI
-Attitudes
-Behaviors
-Characteristics

-0 = Does Not Apply
-1 = Rarely
-3 = Occasionally
-4 = Frequently
-5 = Always

-Authoritarian Personality:
-Variable: What we need is not political programs and resolutions, but strong and devoted leaders whom the people entirely trust.
-SD = 1
-D = 2
-SoD = 3
-SoA = 4
-A = 5
-SA = 6
-DK = 0

-It is: mutually exclusive, exhaustive, ordered, assigned standardized numerical value

-Did you feel that you could tell someone about the bad thing that happened to you?
-1 = Definitely Yes
-3 = In the Middle
-5 = Definitely No
-0 = No Answer

-Notes:
-1. The standardized numerical values assigned to the attributes are arbitrary
-2. There is no true zero point

-*Facebook Experiment
-Used a 5-Point Likert Scale (1 = No at all; 5 = to a great extent)
-Answer to which degree they were experiencing enthusiasm, happiness, loneliness, enjoyment of life, depressiveness, sadness, decisiveness, anger, and worry (all under “emotions”)

-Want to make sure you have equal number of positive and negative answers

14

Ratio

-A ratio variable measures attributes that are:
-1. Mutually exclusive
-2. Exhaustive
-3. Ranked according to some logical order
-4. Assigned a true numerical value
-5. Include a true zero point (zero is zero -- it’s not arbitrary)
-Most precise; easiest to create
-The “level” of the question/variable is determined by the answers we ask respondents to select -- the attributes
-*Some people say ordinal and interval are same/similar

-Indicator: Income
-Ratio Variable: What was your net income for the 2015 fiscal year?
-$__________ (Attributes)

-Above question is mutually exclusive, exhaustive, logical order assigned true numerical value, include true zero point

-Ex:
-How many children do you have?
-How many traffic tickets in the past month?
-How many cigarettes a day?
-How many years of college?

-Facebook Experience:
-Looking at intensity of Facebook use: Number of friends; Daily time on Facebook

15

Matrix Questions?

-Closed-ended questions which have one or more row items which need to evaluated by respondents on the same column items

-Multiple choice questions represented in a grid format

-Like surveys taken for UNLV

16

Contingency Questions?

-Ex: Have you gained or lost significant weight in the past 2 weeks? (+/- 15% of body weight).
-No (If no, go to question 5)
-Yes
-If yes, how many pounds have you gained or lost in the last 2 weeks? ________

17

Rules to Develop Quality Questions/Attributes?

-1. Good grammar is essential

-2. Avoid jargon (use 8th grade level)

-3. Questions should be precise and short

-4. Avoid vagueness about attitudes, beliefs, situations, time periods
-Avoid ”ever,” “never”
-In the past 10 years -- avoid
-Ex: Have you lied? → Have you lied over the last 2 weeks?

-5. Avoid questions that fail to make a meaningful difference

-6. Ask single-issued questions
-Ex of bad at following this rule: How concerned are you that President Bush is not doing enough to get Americans back to work, create more jobs, and get the economy moving again?

-7. Avoid Double-Negatives
-Ex that doesn’t follow this rule: In general, parents should avoid ignoring the situations when their children refuse to obey. (too many negatives)
-Strongly agree, agree, disagree, strongly disagree, not sure
-Ex that doesn’t follow this rule: We should ignore others’ failure to accept group recommendations
-Strongly agree, agree, disagree, strongly disagree, not sure

-8. Avoid questions about future intentions

-9. Ask questions that measure what we intend to:
-Characteristics
-Orientations
-Actions
-Ex: Variable: Lying
-Level: Ordinal
-Question 1: It’s okay to lie from time to time in order to keep the peace at home.
-Strongly agree, agree, not sure, disagree, strongly disagree
-Question 2: In the week, how frequently did you lie when discussing your accomplishments?
-Very rarely, infrequently, frequently, very frequently

-10. Avoid biases
-(A) Loaded Language
-Ex: Do you favor or oppose x?
-Ex: How concerned are you that massive budget deficits... will inevitably result in drastic cuts in…
-(B) Leading Questions
-Ex: Don’t you agree that…
-Ex: Most people agree that…
-(C) Restricted Response Range
-Excellent (+), good (+), fair (+), poor (-), undecided
-Ex: Three positive answers, three negative answers

-11. Limit number of attributes: 5-7
-Ex: How often do you miss going online when you cannot do so?
-Most of the time, very frequently, frequently, very often, often, not very often, sometimes, rarely, very rarely, almost never, never (too many answers)

-12. Orient to Respondents
-Remember that your respondents may be very different from you in terms of age, social class, educational level, typical day, marital status, physical abilities, religiosity, etc.
-Ask questions that are likely to be relevant to them
-Ex: Different language and questions for children, professors, prisoners, etc.

-Refers to decisions the researcher makes about how to collect observations

18

What is validity? Why is it important?

-A term describing a measure that accurately reflects the concept it is intended to measure
-Ex: Your IQ would seem a more valid measure of your intelligence than the number of hours you spend in the library would

-Do my questions/variables really measure what they are supposed to/what we say we are measuring?

-Do they really “get at” the indicators of the dimensions of the concept I am trying to measure?

-Why: to make sure we’re measuring what we want to measure

-5 Types or levels of validity

19

When is a measurement valid?

-Valid when a question/variable measures what it is supposed to measure

-Am I really measuring what I want to measure?

20

What are the various forms of validity?

-1. Face Validity

-2. Content Validity

-3. Internal Validity

-4. Construct Validity

-5. Criterion Related/Predictive Validity

21

Face Validity

-My questions/variables are, by convention, recognized as having something to do with the concept
-Sadness, loss of interest, feelings of emptiness (measure face validity of depression)
-Frequent nightmares? Not face validity
-Compulsive behaviors?

-Show that, by common agreement, they measure the concept they’re supposed to

22

Content Validity

-My questions/variables measure all the dimensions of the concept
-Dimensions of depression (have to measure all dimensions of depression)
-Dealers’ income
-Education
-Testing on “intelligence”

-Show that they measure all the dimensions of my concept

23

Internal Validity

-Variables that measure the same indicator correlate in an expected fashion with each other

-Expected fashion with each other

24

Construct Validity

-My questions/variables correlate in expected ways with other questions/variables related to the concept I am measuring

-Show that they correlate in an expected fashion with other indicators which measure a related concept

25

Criterion Related/Predictive Validity

-My questions/variables correlate in expected ways with a behavior (criterion) which is logically related to them

-Show that you can predict respondents’ behaviors on the basis of how they answer your indicators

26

What is reliability? Why is it important?

-That quality of measurement method that suggests that the same data would have been collected each time in repeated observations of the same phenomenon; quality of repeatability in untruths
-Ex: ”Did you attend religious services last week?” would have higher reliability than question “About how many times have you attended religious services in your life?”

-When it yields the same results on a number of different occasions

-When it yields the same regardless of who is measuring

-Do my questions/variables consistently yield the same results?

-Are they dependable?

-Measurements are reliable when they yield the same results on different occasions, regardless of who is measuring

27

When is a measurement reliable?

-Measurements are reliable when they yield the same results on different occasions, regardless of who is measuring

28

What causes measurements to be unreliable?

-Questions are:
-Too complex
-Too vague
-Irrelevant to respondents
-Influenced by context (ex: given survey too late for doctor, and too early for bank)
-Influenced by researcher

-More valid → less reliable

-More reliable → less valid

29

How can we increase reliability?

-Be careful to ask only about things respondents are likely to know the answer to

-Ask about things relevant to them

-Be clear in what you’re asking

-Test-retest method
-Split-half method
-Using established measures
-Reliability of research workers

30

Nadelson et al.’s article “I Just Don’t Trust Them”

-Ordinal level was their tool (use of reverse coded items)

-Research goals:
-Create a set of items that asses a domain-general level of trust in science and scientists
-Establish the validity of our instrument with experts, using their feedback to refine our items
-Field-test our survey to establish the instrument’s reliability

-”Trust in Science and Scientists”
-Researchers developed Trust in Science and Scientists Inventory
-Assesses of the level of trust in science and scientists
-21-item instrument
-Has a reliability of Cronbach’s alpha of 0.86

-People aren’t trusting science or scientists
-Climate change, vaccination, biological evolution, and genetically modified foods

-Why?
-Popular news stories (or personal perception) of researchers manipulating data, engaging in potentially unethical practices, using questionable methodologies, and withholding results
-Lack of understanding of the fundamental tenets of the nature of science, such as the tentative nature of knowledge and the reliance on empirical evidence to describe the natural world

-A lot of focus on trust (as an attitude)
-Using subjective trust-based decision-making on scientific issues in the public realm (e.g., genetically modified foods, hydraulic fracturing, vaccines, climate change) may be at odds with the objective, data-driven nature of science, and may lead to decisions that stifle the associated scientific development, research, and applications

-People do take credibility into account when determining level of trust; levels of trust based on credibility may come into play with medical procedures, pharmaceuticals, stem cell research, and diagnostic procedures, particularly under conditions in which our knowledge is limited, the conditions are personal, and individuals are potentially more vulnerable.

-Worldviews, perceptions of knowledge, and thoughts about the plausibility of ideas can have a significant influence on our level of trust in others and in their associated institutions

-Argue that items aligned with specific elements of trust may be responded to using other elements of trust or a combination of elements that are not representative of the targeted facet of trust.

-We have found is that trust in science and scientists can have profound influence on policy, funding, and even legislation, and yet trust is a multifaceted construct that includes aspects of emotions, perceptions of credibility, perceptions of trustworthiness, world views, and knowledge.

-They created a valid and reliable instrument

-Goal was to develop the first instrument we are aware of that uses multiple items to assess trust in science and scientists on a general level.

-Used pre-tests

-Shared instrument with five external researchers with expertise in measurement…

-Used preliminary pretest

-Used formal pretest

31

What is sampling? Why do we sample?

-Decisions we make and strategies we use to collect observations about a phenomenon (people, groups, artifacts, behaviors, etc.)

-We always make decisions about what will “count” as an observation. We always sample.

-The scientific approach requires us to be systematic, clear, and precise about our decisions

-Why? Because it is neither necessary nor efficient nor typically feasible to collect all possible observations about a phenomenon

-2 Types:
-Probability
-General term for samples selected in accord with probability theory, typically involving some random-selection mechanism
-Non-Probability

-More important with deductive path

-Ex: Dr. takes blood sample and whatever she finds in sample can be said about all blood for person

32

What are the key characteristics (advantages?) of probability samples?

-1. More representative than non-probability samples

-2. Eliminate biases when selecting elements from sampling frame

-3. Allow us to apply a number of useful statistical laws
-Estimate how close our sample statistics are to the population parameters

-4. Used more typically in quantitative research

-Rules of Thumb:
-1. The more heterogeneous a population, the larger the sample size
-2. Measuring a large number of variables requires a larger sample
-3. 50 cases for each population

33

What are the advantages of probability samples?

-Avoids researchers’ conscious or unconscious biases in element selection
-If all elements in population have an equal (or unequal and subsequently weighted) chance of selection, there is an excellent chance that the sample so selected will closely represent population of all elements

-Permits estimates of sampling error
-Although no probability sample will be perfectly representative in all respects, controlled selection methods permit the researcher to estimate the degree of expected error

34

Types of probability samples?

-1. Simple Random
-Taking the last two digits of every set of numbers

-2. Systematic
-Sampling Frame = 30,000 elements
-Sample Size = 3,000 elements
-Sampling Interval = 30,000/3,000 = 10
-Sampling Ratio = 30,000/10,000 = 1/10
-Taking every 10th on list?

-3. Stratified
-Proportionate and Disproportionate to Size
-Take 1, 4, 7, 10, 13, 16 from magazine set from 1960s, from 1980s, and from 2000s
-Stratified: Systematic Random
-Take from separate set of magazines, but instead of taking 1, 4, and 7 from each, you just number all magazines together and continue counting pattern

-4. Multi-Stage Cluster
-We use this sampling strategy when a sampling frame does not exist
-Step 1: List of all first-level clusters
-Step 2: Sample from first-level clusters
-Step 3: List of all Level 2 clusters
-Step 4: Sample from Level 2 clusters
-Sample of all homeless people in Las Vegas in 2019
-Sample of all the patients who were hospitalized in Vegas hospitals in 2018
-Sample of all Vegas high school seniors who graduated in May 2019

-Cluster (review questions): select two random groups and collect data on all of them

35

What is representativeness? When is a sample representative?

-That quality of a sample of having the same distribution of characteristics as the population from which it was selected. By implication, descriptions and explanations derived from an analysis of the sample may be assumed to represent similar ones in the population. Representativeness is enhanced by probability sampling and provides for generalizability the use of inferential statistics

-Representative when it produces the characteristics existing in the population from which it is drawn

36

What do probability laws tell us about the relationship between sample statistics and population parameters? (I’m not sure.)

-It answers the following questions:
-Is my sample representative?
-Can I generalize from my sample to the population from which it is drawn?
-How close is my sample statistic to the population parameter?
-How confident am I that the statistics (variables) I measure in the sample represent the parameters existing in the population from which it is drawn?

-In order to be able to generalize from the sample to the population from which it is drawn, the sample must be representative

-It must reproduce the characteristics existing in the population from which it is drawn

-The key to create a probability sample is to select its elements through a process of random selection
-Every element of the sampling frame has an equal non-zero chance of being selected for the sample

37

What are non-probability samples?

-Selection of observations is not random

-Any technique in which samples are selected in some way not suggested by probability theory

38

What are the different types of nonprobability samples? When do we use them?

-1. Accidental/Convenience:
-The researcher collects observations in terms of convenience, accessibility, etc.
-Ex: malls, airports, etc.
-Ex: FB experiment
-We Have to Take These Guys Out

2. Purposive/Judgemental
-Researcher collects observation on the basis of his/her knowledge of the phenomenon and objective
-Ex: Sampling members in ethnography, interview research, sampling texts in historical research and textual analysis
-Ex: We Have to Take These Guys Out (focus on men)

3. Snowball
-The respondents suggest who/what else to sample

4. Quota
-The researcher collects a representative number of observations, but the process is not necessarily random (representative, but not random)
-Ex: of multiplying liberal arts by F by work (65 x 40% x 30% = 8)

-5. Theoretical Sampling

39

What are the advantages, disadvantages, and limitations of non-probability samples?

-Avantages:
-Good for qualitative research projects
-Get inside look from informants
-Good for times when probability sampling wouldn’t be appropriate even if it were possible

-Disadvantages:
-Informants’ marginality bias
-Marginal status may also limit their access (and hence yours) to different sectors of the community you wish to study
-Simply because they’re the ones willing to work with outside investigators, informants will almost always be be somewhat “marginal” or atypical within their group

-Limitations:
-Not accurate
-Not precise representations of populations

40

Population

-Theoretical construct
-Americans, the middle class, UNLV students, TV commercials, blogs, etc.

-Even though we ultimately want to make claims about the population, it cannot be studied

41

Target/Study Population Sampling Frame

-Specifications about who/what will be studied

-Ex: Bostonians: Boston residents aged 18-54, who were American citizens at the time of the study, and possessing English language proficiency

-Ex: Commercials: All commercials aired between September and December 2019 on ABC, CBS, NBC, FOX channels between 12 am and 11 pm

-Defining the target population typically entails deciding on criteria for membership (age, time, place)
-Ex: UNLV Students: All individuals registered as students and attending classes on the UNLV main campus and online during the 2019 Fall semester

42

Sampling Frame

-Concrete representation of the target population

-List of people who meet criteria

-ON EXAM

43

Sample

-Subset of the sampling frame

44

Parameter

-Summary measure of a variable in the population

-Ex: Age, income, GPA, gender, vote
-Cannot be known

45

Statistics

-Summary measure of a variable in the sample

-Ex: Age, income, GPA, gender, vote
-Can be known

46

Element

-Unit of the sample about whom/which we collect observations

47

Flores - UNLV Pros Article

-91 Percent of UNLV Professors, but it only included 125 professors from mostly from Liberal Arts

-Not representative of all professors

-”UNLV Professors”
-An overwhelming majority of UNLV professors are registered Democrats, according to a
recent study, leading some students to believe that their classroom experience is lacking
in intellectual diversity.

-91 percent of UNLV professors studied affiliate themselves with parties on the left of the political spectrum.

-Study also found that some departments at UNLV don't have a single registered Republican teaching courses

-Roots solution: The solution is to do away with tenure, which he believes obstructs the free market

48

Tromholt’s FB Experiment

-It is a non-probability sample
-Accidental/Convenience

-Consisted of 86% women, geographically residing throughout the country

-IV = intensity of FB use
-DV = life satisfaction; emotions

-Classical Experiment
-We hypothesize that X (IV) will cause a change in Y (DV 1) and Z (DV 2)
-X: IV = stop using FB for a week
-Y: DV 1 = Emotional well-being
-Z: DV 2 = Life Satisfaction

49

We Have to Take These Guys Out

-It is a non-probability sample
-Accidental/Convenience
-Purposive/Judgemental (focus on men)

-Interviewed 73 men and women previously incarcerated in California prisons

50

What are experiments? When are they appropriate and useful?

-Step 1: Pretest both groups on variables

-Step 2: Administer treatment to experimental group

-Step 3: Post-test both groups on same variables and compare

-Effect of treatment? There should be no difference in control group if done correctly, but there should be a difference in experimental group if treatment does have effect

-When to use it:
-To test hypotheses (deductive)
-About causality
-Among a small number of variables

-1. Two groups (experimental and control)

-2. A pre-test of variable(s) in both groups

-3. A stimulus (treatment) only in experimental group

-4. A post-test of variable(s) in both groups

-1. Does X cause change in the variable (O1 and O2)?
-2. What are x’s effects on the variable?
-3. Is it true that x will cause a change in the variable?

EG O1 x O2
CG O1 O2

-Types:
-Classical Design
-Pre-Experimental Designs
-Field Experiments
-Natural Experiments
-Evaluation Research
-Online Experiments

51

Elements of the Classical Scientific Design

-1. Two groups (experimental and control)

-2. A pre-test of variable(s) in both groups

-3. A stimulus (treatment) only in experimental group

-4. A post-test of variable(s) in both groups

-1. Does X cause change in the variable (O1 and O2)?
-2. What are x’s effects on the variable?
-3. Is it true that x will cause a change in the variable?

EG O1(pretest) x(stimulus) O2(post-test)
CG O1 O2

-Ex: We hypothesize that X (IV) will cause a change in Y (DV 1) and Z (DV 2)
-X: IV = stop using FB for a week
-Y: DV 1 = Emotional well-being
-Z: DV 2 = Life Satisfaction

-Remember: conceptualize and operationalize both dependent and independent variable

52

Different Types of Pre-Experimental Designs

-Experimental designs that lack pretests and/or control groups

-1. Static Groups Comparison
-Experimental and control group
-No pretest
EG x O2
CG O2
-Ex: Bobo Doll Experiment

-2. One Group Pre-Test Post-Test Only:
-Pre-test, post-test
-No control group
EG → O1 (number of tickets before) x(stimulus) O2(number of tickets after)
-Ex: Milgram

3. One Shot Case Study
-No pre-test
-No control group
EG x O2
-Ex: Milgram, Rosenhan, and Rosenthal & Jacobson

53

How do we distribute subjects to experimental and control groups?

-1. Random Selection

-2. Matching
-Single Men, Under 30 → 4 into control group, 4 into experimental group
-Note:
-1. The two groups must be representative of each other, not of the general pop
-2. Sometimes, we do not know what are the variables according to which we should match both groups

54

Variations

-1. Additional Experimental Groups
EG O1 x O2
EG2 O1 2x O2
CG O1 O2

-2. Change the IV
EG O1 x O2
EG 2 O1 2x O2
CG O1 O2

3. Post-Test on Multiple Occasions
EG O1 x O2 O3 O4
EG2 O1 2x O2 O3 O4
CG O1 O2 O3 O4

55

What are double-blind experiments? Why are they useful?

-Neither the observers nor the subjects know which is the control or experimental group
-Compensation
-Experimenter’s expectations
-Use it to prevent compensation and experimenter’s expectations

56

What are the sources of internal validity? What do they mean?

-Are the changes observed in the DV (from pre-test to post-test) caused by the IV (x: the stimulus) or by other factors?

Sources:
-1. Design:
-1. Testing (person may be alarmed that they’re being tested)
-2. Instrumentation
-3. Selection Bias (not careful in randomly selecting)
-4. Compensation (feel bad for CG and give them more attention)
-5. Causal Order
-6. Pygmalion Effect/Experimenter’s expectations (Hans horse: Hans saw farmer’s reaction; understand what experimenters want to see)

-2. External
-1. History (ex: measuring effect of movie, but day before watching movie, environmental catastrophe happened, so results are influenced by history)

-3. Subjects
-1. Maturation (people change/evolve)
-2. Attrition (some subjects leave -- those who stayed vs. left have certain characteristics)
-3. Contamination/Diffusion (EG person goes to bathroom and meets someone in CG and they talk, which changes behavior)
-Compensatory Rivalry (members of CG work extra hard to go against researcher)
-Demoralization (CG upset they’re not getting any rewards)
-4. Statistical Regression
-5. Hawthorne Effect (subjects feel special because they’re part of an experiment)

57

What is the source of external validity? What does it mean?

-Refers to the possibility that conclusions drawn from experimental results may not be generalizable to the “real” world

-Sources:
-Generalizability of experimental findings is jeopardized if there’s an interaction between the testing situation and the experimental stimulus (ex: muslim history film with classic experimental design)
-Solomon Four-Group Design (solution)
-Posttest-Only Control Group Design (solution)

-Are the changes observed in the dependent variable (from O1 to O2) caused by
-the simulus (x) or
-the interactive effect of pre-testing and stimulus (O1 and x)?

58

What is the Solomon’s Four Group Design? What are its benefits?

-Addresses the problem of testing interaction with the stimulus

-Involves 4 groups of subjects, assigned randomly from a pool

-Classical experiment runs the risk that pretesting will have an effect on subjects, so the Solomon four-group design adds experimental and control groups that skip the pretest. Thus, it combines the classical experiment and the after-only design (with no pretest)

-Benefits:
-Rules out interactions between testing and the stimulus
-Provides data for comparisons that will reveal how much of this interaction has occurred in a classical experiment (allows researchers to review and evaluate the value of any prior research that used the simpler design)

EG1 O1 x O2
CG2 O1 O2 (from EG1 to CG2, 30%)
EG3 x O2
CG4 O1 O2 (from EG3 to CG4, 10%)

59

What are natural experiments? Characteristics? Examples?

-Situations where a stimulus is introduced in society or a group thereof, and which experimenters do not control
-Natural catastrophes
-Presidential debates
-New laws
-Terrorist acts, shootings, riots
-Technological accidents
-Information manipulation

-May raise validity problems

-Still requires ingenuity and insight

-Can still have control and experimental groups
-Workers in plant that had problem vs. workers in plant that didn’t have problem

60

What are field experiments? Characteristics? Examples?

-Experiment conducted outside the lab, in real life, in real world

-Origins

-Characteristics
-No precise pre-test
-No precise post-test
-No control group

-Pygmalion; My Fair Lady - tendency to see in others what we’ve been led to expect
-Rosenthal and Jacobson experiment (telling teachers which students are promising → teachers thus give them more attention → of course, those students improved

61

What is evaluation research? Characteristics? Examples?

-Research undertaken for the purpose of determining the impact of some social intervention, such as a program aimed at solving a social problem; purpose is to evaluate the impact of social interventions, such as new teaching methods or innovations in parole

-Characteristics:
-Main operational aspects: measurement, study design, and execution
-Form of applied research (that is, it’s intended to have some real-world effect)
-Methods that can be used: surveys, experiments, and so on
-Appropriate whenever some social intervention occurs or is planned
-Must be able to operationalize, observe, and recognize the presence or absence of what is under study
-Not a method itself, but rather one application of social research methods!

-Types:
-Experimental Designs
-Quasi-Experimental Designs
-Qualitative Evaluations

-Dependent variable (O1) = a condition judged undesirable

-Stimulus (x) = intervention

-Post-test (O2) = change in undesirable conditions

-Ex:
-Head-Start
-Scared Straight
-D.A.R.E.
-Police Organization Research
-PTSD Training

62

What are online experiments? Characteristics? Examples?

-In labs

-Virtual Spaces

-Social Media
-Ex: FB experiment

63

What are the strengths and weaknesses of experiments?

-Strong validity

-Weak reliability

-Would the stimulus reliably produce the same effects outside of the lab?
-Milgram’s research in New Haven office

64

What is survey? When is it appropriate and useful?

-In a typical survey, the researcher selects a sample of respondents and administers a standardized questionnaire to them

Uses:
-To study large number of dispersed individuals; best method available to social researcher who is interested in collecting original data for describing a population too large to observe directly

-Descriptive, exploratory, explanatory

-Large number of variables

-Inductive and deductive

-Strengths:
-Multi-Purpose:
-It can measure a wide variety of issues and provide information for a wide variety of interests:
-Sociological issues
-Political Voting
-Marketing
-Health
-Transportation
-Education
-Government assessing community and state needs
-State agencies assessing programs
-Legal Claims
-Media
-Efficient: It can
-Collect large number of information
-About large numbers of people
-At relatively low costs
-Cost of measuring many variables is not that much higher than measuring a few
-Quickly
-It guarantees the standardization of measurement
-Large number of dispersed people have been measured/observed in the same way

65

What does “the standardization of measurement” mean?

-Large number of dispersed people have been measured/observed in the same way

66

How can we enhance the quality of a survey?

-(1) Overall organization
-Maintain constant focus
-(a) Clear idea of research questions
-(b) Keep population being measured in mind
-(c) Integrated whole

-(2 and 3) Questions and Sections
-(A) Group questions into logical sections (ex: social life, information access, religiosity, attitudes, demographics -- each section has 4 questions)
-a. Each section serves a purpose and is related to the research question
-b. Each question within each section serves a purpose and is related to the research question
-(B) Organize the order of the questions (ex. Asking about happiness in general and then marital life vs. vise versa)
-a. Interesting sections first
-b. Uncomfortable sections middle or last
-c. Demographic section last
-d. Introductions to and transitions between sections
-e. Concluding statement (Thank you, etc.)
-(C) 14 Rules

-(4) Aesthetics

-(5) Cover Letter

67

Important components of questionnaire-building?

-Design → Pre-Delivery → Delivery → Post-Delivery

-1. Overall organization
-2. Questions
-3. Sections
-4. Aesthetics
-5. Cover Letter

68

Design

-Questionnaire: Tool that measured people at distance

-Sole medium of communication between researcher and respondent

-(1) Overall organization
-Maintain constant focus
-(a) Clear idea of research questions
-(b) Keep population being measured in mind
-(c) Integrated whole

-(2 and 3) Questions and Sections
-(A) Group questions into logical sections (ex: social life, information access, religiosity, attitudes, demographics -- each section has 4 questions)
Each section serves a purpose and is related to the research question
Each question within each section serves a purpose and is related to the research question
-(B) Organize the order of the questions (ex. Asking about happiness in general and then marital life vs. vise versa)
-a. Interesting sections first
-b. Uncomfortable sections middle or last
-c. Demographic section last
-d. Introductions to and transitions between sections
-e. Concluding statement (Thank you, etc.)
-(C) 14 Rules

Questions:
A. We ask survey questions to many people, not just one
B. Our survey questions/attributes must be understood in the same way by all respondents, who may be very different from each other
C. Our survey questions/attributes must be understood in the same way by all respondents, who may be very different from us
D. When we deliver our questionnaire by mail or online, we can’t rephrase a question in the respondent does not understand
E. When we deliver our questionnaire face-to-face or by phone, respondents do not know us and cannot catch nuances of expression

-(4) Aesthetics

-(5) Cover Letter

69

Rules of questionnaire aesthetics?

-1. Provide generous space

-2. Organize different sections on different pages

-3. Distinguish questions and answers
-Indents
-Numbers
-Fonts (different fronts for questions and answers)
-Ex: Over the past two weeks, I have noticed I have more difficulty falling asleep
(1) Yes
(2) Not Sure
(3) No
-Answers are indented; answers have numbers; answers are in a different font

-4. Provide visual aid for contingency questions
-Ex: Do you smoke marijuana?
-a. Yes
-b. No. If no, go to question 6 (include arrow to number 6)

-5. Provide clear instructions
-Answering
-Returning/submitting the questionnaire
-Self-addressed stamped envelope

70

14 Rules of Quality Questions?

-1. Correct grammar is essential

-2. Avoid jargon

-3. Questions should be precise and short

-4. Avoid vagueness about attitudes, beliefs, situations, time periods

-5. Avoid questions that fail to make a meaningful difference

-6. Ask single-issued questions
-Ex: (bad) How often have you used the following sources for UNLV information such as courses, events, requirements, and the like? (Circle one number for each row)

-7. Avoid double-negatives
-Ex: Most of the time, parking is NOT a problem…
-Ex; I usually do NOT REFUSE requests that are NOT made at the last minute

-8. Avoid questions about future intentions

-9. Ask questions that measure what we intend to:
-Characteristics
-Orientations
-Actions

-10. Avoid biases
-a. Loaded language (ex: Should the US intervene in the world for a vietnam-like situation? Should the US intervene in the world to stop a communist takeover?
-b. Leading questions (ex: Don’t you agree…. Most people agree…); ask neutral questions
-c. Restricted response range (ex: three positive answers and only one negative one)

-11. Mind the fence-sitters
-When forced to make a choice, 10-20% of respondents who do not have a strong opinion will choose an explicit position

-12. Mind the floaters:
-30% of respondents who know nothing about a topic will choose an explicit answer in the absence of a “don’t know” option

-13. Add interpretive questions to make sure we know
-What our respondents mean
-They understand the question
-Ex: Do you think your emotional state affected your driving on the day of the violation? Yes (explain). No. Interpretive Question: What else was going on on the day of the violation? ___________

-14. Induce truthful answers
-People should want to answer truthfully
-(a) Frame deviant behavior as unexceptional
-(b) Frame problematic behavior as benign
-(c) Ask the same question differently, in different sections
-Respondents are likely to overreport
-Social desirable behaviors
-Being informed
-Fulfilling moral responsibilities
-Having a happy life
-Respondents are likely to underreport
-Socially undesirable behaviors
-Being uninformed
-Failing to fulfill moral responsibilities
-Having an unhappy life

71

Types of Bias in Questions?

-Loaded language
-Ex: Should the US intervene in the world for a Vietnam-like situation? 18% say yes
-Ex: the US intervene in the world to stop a communist takeover? 33% say yes

-Leading questions
-Don’t you agree…
-Most people agree…

-Restricted response range
-Three positive answers, one negative

72

How to increase the likelihood that respondents will answer truthfully?

-14. Induce truthful answers
-People should want to answer truthfully
-(a) Frame deviant behavior as unexceptional
-(b) Frame problematic behavior as benign
-(c) Ask the same question differently, in different sections
-Respondents are likely to overreport
-Social desirable behaviors
-Being informed
-Fulfilling moral responsibilities
-Having a happy life
-Respondents are likely to underreport
-Socially undesirable behaviors
-Being uninformed
-Failing to fulfill moral responsibilities
-Having an unhappy life

73

What are pre-tests?

-The measurement of a DV among the subjects; subjects are measured in terms of a dependent variable; before being exposed to IV

74

What are different kinds of pretest? How do we conduct them? When do we conduct them? With whom do we conduct them? Why do we conduct them?

Types:
-1. Preliminary Pretests
-Friends and colleagues
-Ex: Nadelson article sharing instrument with researchers
-2. Formal Pretests
-Individuals similar to respondents
-Ex: Nadelson article administering it to 75 undergraduate college

-Preliminary Pretest → Formal Pretest → Delivery to Respondents → Follow-Ups

75

What do we look for when analyzing the pretests?

-Look for:
-Skipped questions
-Questions answered similarly by all
-Questions that weakly correlate with others
-Response sets
-Use reverse-coded items (used in Nadelson article

76

What aspects of a cover letter are likely to increase respondents’ willingness to participate?

-Credible
-17% increased response rate for government and university

-Personalized
-7% increased response rate

-Interesting

-Voluntary Participation

-Confidential or Anonymous

-Identify Yourself and Research Purpose
-Name, affiliation, contact info

-Appeal
-Science, community, society, respondents

-Incentive

77

What are the main forms of survey delivery?

-Mail

-Face-to-Face

-Phone

-Online

78

Advantages and Disadvantages of mail delivery?

-Advantages:
-Quick
-Cheap
-Sensitive Questions
-Large numbers
-No researcher’s effects

-Disadvantages:
-No control over sequence
-No control over identity of respondent
-Low return rate (50%)
-No probing or explaining
-No complex questions

-Least researcher’s effect

-Not expensive

-Most appropriate for sensitive questions

-Low response rate (50%)

-Least control over the order of questions answered and the identity of the respondent
-Standardization of measurement?

79

Advantages and Disadvantages of face-to-face delivery?

-Advantages:
-Immediate entry of answers
-Least skipped questions
-Allows for probing
-Allows for complex questions
-Highest return rate (80%)
-Allows for recording of additional information

-Disadvantages:
-Strange situation
-No sensitive questions
-Neutrality (tape-recorder)
-Warm technician
-Researcher’s effect
-Most expensive
-Requires training and traveling

-Remember:
-Make contact (letter, phone)
-Non-threatening questions first

-Has highest response rate (80%)

-Is most appropriate for complex questions

-Is most expensive

-Is the least reliable one (researcher’s effect)
-Tool guarantees the standardization of measurement

80

Advantages and Disadvantages of phone delivery?

-Advantages:
-Quick
-Cheap
-Some probing
-Some sensitive questions
-Immediate entry into database
-Few skipped questions

-Disadvantages:
-5% of population don’t have a phone
-9% response rate
-Increasing number of screening devices
-Telemarketing/Suspicious Calls
-Time and representativeness issues
-Distractions
-Training

-This mode of delivery is meeting most research criteria, but has the lowest response rate

81

Advantages and Disadvantages of online delivery?

-Advantages:
-Cheap(est)
-Quick(est)
-Control over sequence
-Control over timing
-Immediate data entry and analysis
-Hybrid (chat)

-Disadvantages:
-Low return rate (30%)
-Representativeness
-Distraction
-Fatigue
-New context, new risks

-Suffers from the same weaknesses as mail and has an even lower response rate

-Mail/online modes of delivery present the greatest risk of misinterpretations

82

What are important tensions between these different modes of delivery?

-Phone mode of delivery is meeting most research criteria, but has the lowest response rate

-Face-to-face mode of delivery:
-Has highest response rate (80%)
-Is most appropriate for complex questions
-Is most expensive
-Is the least reliable one (researcher’s effect)
-The tool guarantees the standardization of measurement

-Mail mode of delivery is:
-Least researcher’s effect
-Not expensive
-Most appropriate for sensitive questions
-Low response rate (50%)
-Least control over the order of questions answered and the identity of the respondent
-Standardization of measurement

-Online mode of delivery suffers from the same weaknesses as mail and has an even lower response rate

-Mail/online modes of delivery present the greatest risk of misinterpretations

83

What is a follow-up? How do we conduct it? When do we conduct it? Why do we conduct it?

-Post-delivery

-First one:
-2-3 weeks after first mailing
-Send letter and questionnaire

-Second one:
-2-3 weeks after first follow-up

-Diminishing return with each follow-up

-Why: try to get more responses in; to remind them to complete it; try to get as many responses back as possible

84

What information do we need to provide when reporting survey results?

-1. Sponsoring organization

-2. Date on which survey was conducted

-3. Population that the sample represents

-4. Sample frame

-5. Sampling method

-6. Sample size

-7. Method of delivery

-8. Exact working

-9. Response rate

-10. Missing information

-Don’t need to know?

85

Strengths and weaknesses of survey research?

-Strengths:
-Strong reliability (but weak validity)
-Relatively cheap
-Produces results relatively quickly
-Can be used to measure a wide variety of issues

-Weaknesses:
-1. Communication patterns
-2. Words are context-dependent
-3. Context of response
-4. Answers created by/because of survey
-5. Respondents’ point of view
-6. Researcher’s non-intervention
-7. Assumption of correspondence
-Tool guarantees standardization of measurement
-a. All respondents understand questions similarly
-b. All respondents understand answers similarly
-c. All respondents understand both questions and answers as the researcher
-8. Validity (questions of validity - Am I really measuring what I am measuring?)
-Cross-cultural (ex: of culture in Beirut)

86

What is analysis of existing statistics? When is it useful? How to conduct it?

-Utilizing already existing statistics to analyze them in new ways
-Cross-sectional research
-Longitudinal research
-Cross-national research

87

Strengths and weaknesses of analysis of existing statistics?

-Advantages:
-Cost (doesn’t cost anything) and time (doesn’t take a lot of time)
-Sometimes, provides access to high quality data
-More time for data analysis
-May offer new insights

-Disadvantages:
-No control over data quality or presence of key variables that interest the user

88

What is secondary analysis? When is it useful? How to conduct it?

-Assess unexamined relationships between different variables
-Cross-sectional research
-Longitudinal research
-Cross-cultural research?

-Analyze them in new ways
-SDA-UC BERKELEY

89

Strengths and weaknesses of secondary analysis?

-Strengths:
-Cost and time
-High quality data
-More time for data analysis
-Reanalysis may offer new interpretations
-Wide obligations of social researcher
-Making full use of respondents' contributions
-Mining data to full extent

-Weaknesses:
-Lack of familiarity with data/variables
-Complexity of the data
-No control over data quality or presence of key variables that interest the user

90

What is unobtrusive research - Purposes, Units, Focus, Time Dimension, Sampling, Types

-Research methods in which there is no interaction between the researcher and the data

-Purposes:
-Exploratory
-Descriptive
-Explanatory
-Criticism

-Units:
-People (alone or in interaction)
-Artifacts/Texts
-1. Audio (all)
-2. Visual/Pictorial (images, engravings, postcards, websites)
-3. Audiovisual
-4. Written (books, newspapers, diaries, speeches, anything written)
-5. Concrete Objects
-6. Traces
-Erosion (signs of wear and tear on objects)
-Ex: World soil degradation
-Accretion (accumulation of objects; what people leave behind)
-Ex: Graffiti; pile of cigarettes; categories of garbage we leave behind
-7. Built Spaces
-Ex: Airports; churches

-Focus:
-Characteristics
-Orientations
-Effects

-Time Dimension:
-Cross-Sectional (past or present)
-Longitudinal (trend, cohort, panel)

-Sampling:
-Probability
-Non-Probability
-1 Single Text

-Types:
-Quantitative: Manifest Content Analysis (Deductive or Inductive)
-Qualitative: Latent Content Analysis (Inductive)

-Can be quantitative or qualitative
-Can be deductive or inductive

-Ex: Lee’s article “Mass Shooting and Media Contagion Theory”?

91

Manifest content analysis? When is it useful? How to conduct it?

-Technique for the objective, systematic, and quantitative description of the manifest/overt content of a communication
-”Objective” = not subjective
-”Systematic” =
-”Quantitative” = just numbers
-”Manifest/Over Content” = systematically analyze; not about meaning

92

Sources of data in unobtrusive research?

-Units:
-People (alone or in interaction)
-Artifacts/Texts
-1. Audio (all)
-2. Visual/Pictorial (images, engravings, postcards, websites)
-3. Audiovisual
-4. Written (books, newspapers, diaries, speeches, anything written)
-5. Concrete Objects
-6. Traces
-Erosion (signs of wear and tear on objects)
-Ex: World soil degradation
-Accretion (accumulation of objects; what people leave behind)
-Ex: Graffiti; pile of cigarettes; categories of garbage we leave behind
-7. Built Spaces
-Ex: Airports; churches

93

How to distinguish between manifest content analysis and latent content analysis?

-Manifest:
-Quantitative
-Deductive or inductive

-Latent:
-Qualitative
-Inductive

94

Sampling in unobtrusive research?

-Sampling:
-Probability
-Non-Probability
-1 Single Text

95

Forms of operationalization in manifest content analysis?

-Nominal:
-Is a particular word/theme/image/metaphor quality present or absent?
-Does a text support or oppose?

-Ordinal
-Intensity of word/theme/image/metaphor/quality
-Add weight to the word (ex: love = strong positive; categories by level of intensity

-Interval
-Quantification of intensity (add number value)
-Quantification of complexity of words/texts/sounds
-Ex: love = strong positive = 6

-Ratio:
-Frequency
-Space and time

-Always record base number
-Ex: 40 out of how many?
-Just reporting, not interpreting

96

Strengths and weaknesses of manifest content analysis?

-Strengths:
-1. Cheapest
-Can be done at home; can download software (that may cost money) to code information
-2. Democratization of what counts as legitimate “texts”
-3. Allows for endless improvement
-4. Universe of potential data
-Has strong validity AND strong reliability

-Weaknesses:
-1. Meaning
-Frequency does not equal significance
-2. Indexicality of meaning
-Meaning of word depends on context
-3. Relevance: Reader’s interpretations

97

Which of the following strategies should you use in order to support your claim that your indicators have content validity?
(A) Show that, by common agreement, they measure the concept they’re supposed to
(B) Show that you can predict respondents’ behaviors on the basis of how they answer your indicators
(C) Show that they measure all the dimensions of my concept
(D) Show that they correlate in an expected fashion with other indicators which measure a related concept

-C

98

When is a measure reliable?

-(E) C and D

-When it yields the same results on a number of different occasions

-When it yields the same regardless of who is measuring

99

Which of the following characterize(s) the work of conceptualization?

-(D) All of the above

-1. Providing a working definition of the concept

-2. Developing the various dimensions of the concept

-3. Providing the indicators of the concept

100

A survey questions asks the following question to students who have just concluded their SOC 403 class:
“In general, how do you rate the efficiency of the course instructor and lab assistant? (Excellent _____, Good _____, Don’t Know/No Opinion ____), Poor ____, Very Poor _____)”
Which important recommendation regarding question-writing does this question violate?

-(A) Questions should measure only one issue

101

Which of the following questions measures marijuana consumption at the ordinal level?

-(D) How frequently do you smoke marijuana? Check one
-Daily
-Weekly
-Monthly
-A few times a year
-I never smoke marijuana

102

Iraq Water Supply
Water in home with safe sources
-Daily Problems
-Problems from time to time
-Weekly problems
-Almost no problems

-Ordinal

103

Very Poor = 1
Poor = 2
Neither Poor nor Good = 3
Good = 4
Very Good = 5

-Interval

104

National Expenditure on Health - International Comparison, 2003 (Provisional Data). Countries along bottom and Percentage of the GDP, starting at 0 along side

-Ratio

105

What is the proper term designating the summary measure of a variable in the population?

-(A) A Parameter

106

Why are probability laws useful?

-(A) They help us assess how close our statistics are to the population parameters

107

What is the necessary criterion to develop a probability sample?

-(D) Its elements must be randomly selected

108

Which of the following samples is/are not (a) probability one(s)?

-(E) C and D

-Quota sample
-Need to select certain number from each group to participate

-Purposive sample
-Purpose (ex: of TA’s study, which used snowball)

109

Wanting to predict how Las Vegas will vote in the next election, a researcher compiles a list of all registered voters in Las Vegas and draws from that list a sample of 2,000 respondents. What is the population being studied in this research?

-(A) Las Vegans

110

Wanting to predict how Las Vegas will vote in the next election, a researcher compiles a list of all registered voters in Las Vegas and draws from that list a sample of 2,000 respondents. What is the sampling frame being studied in this research?

-(D) A list of all registered voters in Las Vegas

111

Wanting to predict how Las Vegas will vote in the next election, a researcher compiles a list of all registered voters in Las Vegas and draws from that list a sample of 2,000 respondents. What is the sample being studied in this research?

-(B) 2,000 respondents

-Population → Sampling Frame → Sample → Statistic? → Element

-Las Vegans → All registered voters in Las Vegas → 2,000 respondents → A single voter

-Sample → Sample Size → Usually means a number

112

Studying trends in movies over the last 3 decades, I first organize all the top-winning movies by decade, and then randomly select 5 movies for each decade. Which kind of sample does this strategy represent?

-(D) Stratified Sample

-30 years of movies → All top-winning movies → Randomly selected

113

Researchers for a marketing firm ask every person who enters the mall on Friday between 10 am and 1 pm how she/she feels about the mall stores and design. What kind of sample does this scenario represent?

-(E) Convenience Sample

114

An experiment measuring whether a documentary movie reduces anti-Muslim prejudice was conducted at Columbia University (NY) between September 9 and September 16 2001. Which source of internal invalidity is most likely to influence the post-tests of experimental and control groups?

-(B) History

115

Which of the following design is useful to reduce the effects of the internal source of invalidity known as “experimenter’s expectation?”

-(C) A double-blind experimental design

116

In order to understand how driving motor vehicles increases aggressiveness, you set up an experiment where you purposefully drive slowly on the freeway while honking randomly. This scenario illustrates?

-(D) A field experiment

-Field → natural environment, but researcher is adding stimuli

-Natural → natural environment, and stimulus occurs regardless of researcher

117

Wanting to support the hypothesis that learning a second language significantly increases problem-solving skills among young children, a researcher devises an experiment whereby experimental group members are pretested on problem-solving skills, then spend a semester studying a second language, and are then post-tested on problem-solving skills. Which kind of experiment has this researcher conducted?

-(C) One group pretest-posttest only

118

In November 2013, the news announced that hackers had stolen millions of credit card records from Target. Has this news prompted a change in credit card use? In order to answer this question, a researcher sends to individuals a questionnaire measuring their credit card use since November 2013. This research is an example of?

-(B) A natural experiment

119

Assuming that the survey research you are conducting includes sensitive questions, which method of administration will be most appropriate?

-(C) Mailed questionnaire

120

In what situation will the face-to-face mode of administration be most appropriate?

-(A) When the questionnaire contains complex questions

121

Which mode of survey administration yields the lowest number of skipped questions?

-(A) Face-to-face

122

What is/are (an) advantage(s) of survey research?

-(E) All of the above

-It is relatively cheap

-It is strong on reliability

-It produces results relatively quickly

-It can be used to measure a wide variety of issues

123

Which of the following characterize(s) manifest content analysis?

-(B) It allows for constant fine-tuning of the measuring instrument