Class 22

Question 1

Do some people work on methods, and others work on theories?

Answer 1

Often go hand in hand

IAT and Person who developed IAT
- The best methods actually challenge /extend / refute theories
- Push for new methods allows for more theories

When applied effectively and thoughtfully, new methodological approaches have the potential to resolve ongoing theoretical
debates and open new areas of research that were previously impossible due to methodological limitations

Question 2

Why do we still have debates over theories in the literature?

Answer 2

Because the methods that we have give us two equally plausible interpretations of this effect

(can’t make progress on debate till new methods)

Question 3

Who gets Nobel prizes?

Answer 3

People who identify a method

And who can show that the method shows things we didn’t know previously

(methods matter!)

A good method can open up theory

Question 4

Explain Method 1: Social Network Analysis

Answer 4

Social network analysis (SNA) is the process of investigating social structures through the use of networks. It characterizes networked structures or individuals in terms of nodes (individual actors) and the ties, edges, or links (relationships) that connect them

Question 5

Social Network Analysis

What is Nodes

What is ties, edges, or links

Answer 5

nodes: (individual actors)
- Entity in a network (dot)

ties, edges, or links: (relationships)

Question 6

Social Network Analysis

What is ‘Directed Edge’?

Answer 6

Edge that has an orientation (e.g., an arrow indicating popularity)

• Ppl might list them as friends more then they list others

Question 7

Social Network Analysis

‘Weighted Edge’

Answer 7

Indicates the strength of a
relationship (e.g., line thickness indicating how close two friends are)

Question 8

Social Network Analysis

Distance

Answer 8

Smallest number of ‘edges’ needed to
connect two nodes

Question 9

Social Network Analysis

Centrality

Answer 9

Importance of a node in the network
(e.g., how many edges a node contains or how many cross-group edges a node has)

Question 10

Social Network Analysis

‘Community’:

Answer 10

Degree to which nodes in a network are connected to one another

Is it interconnected or cliquish

Question 11

What are Advantages of social network?

Answer 11

• Can provide insight that people may not be able to self-report (e.g., who is actually most popular versus who is perceived to be
  most popular).
• Can show the spread of influence or change in a network. (can map change)
• Can identify popular nodes to target for possible interventions.

Question 12

What are some limitations of social network analysis?

Answer 12

• Analyses will only be as good as how much of the network you cover. (you gotta get every middle schooler to map full relationship)
• People belong to multiple networks simultaneously (work, family, school,
  clubs, sports, etc.) so effects in one network may or may not carry over

Question 13

Explain how Paluck, Shepherd and Aronow (2016) used a social network approach to
design and assess the effectiveness of an anti-bullying intervention:

Answer 13

They identified ‘social referents’, meaning kids in a school network that had many connections with other kids, and they reasoned that these social referents would receive more attention from other kids and be looked to for information about group norms. (spread the peace message more easily)

Social networks were created by asking kids to list those other classmates who they had spent time with over the prior week

A random sample of students was selected in each school to receive the intervention, which required meeting with a research assistant every other week and discussing common conflict behaviors at school.
Throughout the intervention, these “seed students” were then encouraged to become the public face of opposition to these conflicts. (Discussion based: kids themselves came to a solution on what the problem was at their school - usually bullying)
(Notably, this intervention lacked an educational or persuasive unit regarding adult-defined problems at school)

Question 14

Paluck, Shepherd and Aronow (2016) used a social network approach to
design and assess the effectiveness of an anti-bullying intervention:

What did this study find?

Answer 14

Schools in the treatment condition saw less conflict than those in the control condition

Less expulsions (meaning less bullying activity)

Question 15

When was the Social Network Analysis study more effective?

Answer 15

The intervention was also more effective when a greater number of the
‘seed’ students in the intervention condition were ‘social referents’

Seed students: Kids with discussion intervention

Social referents: Kids who were central

Question 16

Explain this Social Network Analysis:
Parkinson et al. (2017) used a social network analysis to see how people
encode social networks automatically?

Answer 16

An entire cohort of business school students were surveyed about their
social network (who they like to spend time with)

(business school students love networking)

Take these ppl and show them their cohort mates in an fMRI

• Manipulate:
  
  • 1 social relationship from u (friends)
  • 2 social relationship from u (friends of friends)
  • etc

see if passively viewing diff relationships show up differently

Question 17

What did the study on business students find?

Answer 17

Higher brain activation in 3 areas when they are closer to you

The current results indicate that when encountering familiar individuals, humans may spontaneously retrieve knowledge of where they are located, relative to oneself, in a mental map of ‘social space’

Question 18

What does the business results suggest?

Answer 18

It’s not just race and gender we automatically pay attention to

• But it’s also complex social info we see is encoded quickly: “How are you related to me in my complex social network”

Question 19

Explain Method 2: ‘Distributional Language Analysis’?

Answer 19

Archival analysis (a lot of text)

Looks into which words are most likely to occur with one another

The word “dog”, for example, is represented as more similar to “cat” than to “banana” because contexts containing “dog” are more similar to contexts containing “cat” than to contexts containing “banana”

Question 20

What is the core assumption of Method 2: ‘Distributional Language Analysis’?

Answer 20

Words that co-occur with one another are more likely to have an association within that culture

(and likely have similar meanings)

Question 21

What are some Advantages of distributional language analysis?

Answer 21

• Can provide insight that people may not be able to self-report
  (e.g., what kinds of patterns exist in what we read).
  (can’t self report pattens for every wiki article u read)
• Allows for possible historical analyses for associations that may have existed before more modern measures.
• Easy to analyze change over time or differences between countries. (are implicit associations changing in a culture BEFORE we used IAT)

Question 22

What are some limitations of of distributional language analysis?

Answer 22

• Requires a lot of data (tens of millions of words).
• May be dependent on the type of text you use. (maybe biases/ does it represent the culture)
• Cannot know whether the text is supporting (“men are better at work”),
  reflecting (“the culture believes men are better at work”) or refuting (“it’s impossible that men are better at work”) certain association

Question 23

Distributional Language Analysis

Across 25 languages. Lewis & Lupyan (2020) investigated the association between

1) The strength of the gender-career stereotype in a distributional language
analysis (e.g., how closely “man”/”career” and ”woman”/”home” co- occur vs. the opposite pairing).

2) The strength of the gender-career IAT effect among participants speaking
that language from Project Implicit

What did she find?

Answer 23

We find that the implicit (but not explicit) gender associations of participants in a country is correlated with the gender associations embedded in the dominant language spoken in that country

“Man = Career” societies have higher bias on the IAT

Question 24

What did a follow up study on gender and Distributional Language Analysis find?

Answer 24

follow-up study suggests that these text-based gender stereotypes may
be even stronger in kids’ books

Question 25

What is an important point on Distributional Language Analysis (correlational analysis)

Answer 25

This is only a correlational analysis, so hard to determine causation. To what extent does the language we make or see create implicit gender stereotypes and to what extent do our implicit gender stereotypes create changes in our language?

Question 26

What did a ‘Distributional Language Analysis’ study on gender-career associations across 1. different time periods (19th century vs. 21st century), 2. audiences (children’s vs. adult books) or 3. medium (books vs. spoken words) find?

Answer 26

Found gender-career associations across all three of them

Question 27

How can u use ‘Distributional Language Analysis' on one career?

Answer 27

can also be studied at the individual word or profession (for example, how often the word ‘janitor’ co-occurs with ‘man’ vs. woman’). These profession-level estimates of gender stereotypes from text are strongly correlated with actual gender disparities in employment

Question 28

What is Method 3: ‘Big Data’?

Answer 28

archival analysis that uses a dataset too large to code by hand and instead relies on automated data collection and analysis (for example, webscraping of tweets / police reports) some ‘big data’ approaches more recently are starting to also use experimental designs (for example, making twitter bots that respond to ‘fake news’ stories with either support or criticism

Question 29

What are some Advantages of ‘big data’ approaches?

Answer 29

- Same strengths as archival research but just with greater scope. - Often measures impactful, ‘real-world’ behavior, can have access to large amounts of data that make any findings unlikely to be a “fluke

Question 30

What are some limitations of ‘big data’ approaches?

Answer 30

- Basically, same limitations as archival research but now with less ability to notice possible errors in data collection. (not sure if ur scraper is acting correctly) (hard to see if theres an error) - Analysis dependent on the data that are made available, often hard to get at issues of causality, potential threats to internal validity

Question 31

What was a funny issue that shows a limitation of big data stuff?

Answer 31

Guy coded blogs after 9/11 to see if more negative / positive language was used Found that ppl were using negative words BUT: for an unrelated reasons, many of these sites started going offline (so he was coding "error 404" "network connectivity issues" as negative, cuz error, or issues are negative) Someone found this mistake and bro had to take back his paper Something might contaminate ur data ur unaware of

Question 32

Explain the ‘Big Data’ study on Police Records? (3 year long legal battle to get records)

Answer 32

millions of timepoints related to police behavior in Chicago (merging daily 2.9 million patrol assignments with information like officers’ demographics, stops, arrests, and use of force). (so when someone goes out to work beat and comes back = one data point) FOUND= - Black officers more likely to be assigned to racially diverse neighborhoods. - Non-White officers engaged in less instances of force compared to White officers. - Female officers engaged in less instances of force compared to male officers

Question 33

‘Big Data’ study on Police Records What were these effects were particularly driven by?

Answer 33

Race of target (civilian) these effects were particularly driven by interactions with Black civilians SO: Based on these models, deploying Black officers instead of White officers yields 12.55 fewer stops of Black civilians per 100 shifts, a reduction equal to 39% of typical White-officer volume. By contrast, Black officers make only 1.31 fewer stops of white civilians per 100 shifts than their White counterpart (so black officers aren't stopping more white civilians)

Question 34

Explain the ‘Big Data’ study on Medical Records? (treatment algorithms)

Answer 34

worked with a large hospital to track the treatment records of 50,000 patients over three years Checked: enrollment in a “high-risk care management program”, which provide additional resources to patients with complex health needs Enrollment in the program is primarily determined by a ”risk algorithm” based on patient records, with higher scores leading to automatic enrollment in the program (how sick do we think this person actually is)

Question 35

Explain the ‘Big Data’ study on Medical Records? (treatment algorithms) FINDINGS:

Answer 35

when Black and White patients had the same risk score as determined by the algorithm, Black patients had worse objective health measures. - For instance, for a Black and White patient to earn the same risk score to gain eligibility for the program, Black patients typically had 26% more chronic illnesses Takeaway: White ppl did not need to be as sick to get into the program

Question 36

Explain the ‘Big Data’ study on Medical Records? (treatment algorithms) We know the algorithm was not using race to decide (it was not coded) So how could this be?

Answer 36

they found that it was using health costs (how much patients spend to maintain their health) to predict overall health there is a difference between “receiving health care” (health costs) and “needing health care” (objective health). Treating health costs as a proxy for health needs introduced a racial disparity; Black patients generate less healthcare costs than White patients

Question 37

What did the ‘Big Data’: Medical Records study reveal about bias?

Answer 37

This work represents a key problem identified by work on “algorithmic bias” – even systems designed to ignore race in an effort to remove racial biases may still recreate such biases through reliance on proxy variables

Question 38

Explain Method 4: Large Language Models?

Answer 38

No consistent framework for when to use varies on how much u can use them and when

Question 39

Large Language Models: Example 1 (article in scientistic paper)

Answer 39

Ran study among ppl who believe in conspiracies (moon landing was faked) Have them interact with AI chat bot - About the conspiracy you believe in Programmed to decrease ur belief in the conspiracy After 3 rounds did reduce belief in the conspiracy by 40% (persisted for 2 months)

Question 40

How might the Ai chat bot be applied to intergroup relations?

Answer 40

Discuss with chat bot ur views on some groups (maybe it'll change ur mind)

Question 41

Large Language Models: Example 2 (historical text)

Answer 41

Feed in all historical text about ancient greeks to run studies on (???) Run psychological / behavioural studies on them (why would u do this)