Gillespie H4 (2018) – Custodians of the internet: platforms, content moderation, and the hidden decisions that shape social media Flashcards
Three imperfect solutions to the problem of scale
There is very little data about how large a task platform moderation really is
- Transparency reports à only takedown requests from governments and companies,
they say nothing about how much material is flagged by users, how many posts or
images or videos are removed, how many users do the bulk of the flagging, and so on
* There is too much content and activity to conduct proactive review, in which a moderator
would examine each contribution before it appeared
* Publish-then-filter approach à user posts are immediately public, without review, and
platforms can remove questionable content only after the fact
* Given the enormity of the archives they manage, social media platforms have had to
develop a set of solutions to the challenge of detecting problematic content and behavior
at scale:
- Editorial review à oversight imposed before the content is made available. This is a
resource-intensive approach that is difficult to scale
- Community flagging à it takes advantage of the user base by deputizing users into
the moderation process, but requires a great deal of interpretation and coordination
- Automatic detection of specific kinds of problematic content à like porn, harassment
and hate speech
Editorial review
- Content is reviewed by someone in the employ of the platform, who approves or rejects
content before it is posted - An advantage of this approach is that they oversee the production of content, before it
ever reaches their audience - Mostly used in traditional forms of media à reading an article before the newspaper goes
to the publisher, or watching an episode of a tv-show before it will be aired - The seven-second delay à a technical gap between the moment happening in front of
the camera and when that moment goes out to audiences - This delay reinserts the opportunity for prepublication, editorial review, just long
enough for a censor to ‘bleep’ out certain things - Example of someone running on a soccer field during a match, you don’t want to give
this person any broadcast attention, so in this seven-second delay you have time to
switch the camera, before the situation gets to the audience - Also, used for subtitling
- All iPhones and iPads remain tethered to Apple à Apple can upgrade an app or delete it
remotely, can extract fees from any financial exchanges, and can collect user data from
within them - User can only use and get apps that Apple approves and distributes
- The biggest challenge is how to scale a review process
Community flagging
- For most social media platforms, the amount of material is so immense and relentless,
and the expectation of users that their content should appear immediately is so
established, that prepublication editorial review is impossible - Detection shifts from reviewing everything beforehand, to scouring what is already posted
and available - Flagging mechanism à allows users to alert eh platform to objectionable content
- Using the users is practically convenient in that it divides this enormous task among
many, and puts the task of identifying offensive content right at the point when someone
comes into contact with it - Giving users a say makes the moderation appear more democratic
- Flags are a thin form of expression à they provide little room to express degree of
concern, or contextualize the complaint, or take issue with the rules
Gedownload door: jjverhelst | jjverhelst@outlook.be
Dit document is auteursrechtelijk beschermd, het verspreiden van dit document is strafbaar.
€ 912 per jaar
extra verdienen?
Stuvia - Koop en Verkoop de Beste Samenvattingen - The flag is a fundamentally individualized mechanism of complaint, and is received as
such, but it does not mean it is used that way - Flagging can and does get used in fundamentally social ways:
- Flags are sometimes deployed amid an ongoing relationship (with online bullying,
people may also know each other offline) - Flags can also be a playful prank between friends, part of a skirmish between
professional competitors or rival YouTubers, etc. - Strategic flagging à users will flag things that offend them politically, or that they disagree
with - Arms-length form of oversight à platforms can retain the right to moderate, while also
shifting justification and responsibility to users (solution to flagging problems)
Automatic detection
- Automatic detection à it detaches human judgement from the encounter with the specific
user, interaction, or content and shifts it to the analysis of predictive patterns and
categories for what counts as a violation, what counts as harm, and what counts as an
exception - Automated detection is not an easy task, it’s an impossible one, given that offense
depends so critically on both interpretation and context - There are some limitations to overcome:
- The lack of context
- The evasive tactics of users
- The fluid nature of offense
- Without solving these problems, automatic detection produces too many false positives
- The most effective automatic detection techniques are the ones that know that they’re
looking for beforehand - This is how it works:
- Compare the user’s inputted text against an existing blacklist of offensive words,
either set by the platform, built into the tool by its third-party developer, added by the
user, or through some combination of all three
- There are some limits to automatic detection:
- These tools cannot identify content that has not already bene identified à it must
keep up with new instances - Blacklist tools also encourage evasion techniques à deliberate misdirection, such as
inserted punctuation and misspellings - The filters have a difficult time with words that have multiple meanings and with words
that are associated with adult topics but can be used in other ways - Language is fluid à especially in informal and playful environments
Hashing
a digital image is turned into a numerical string based on the sequence of
colors in the image’s individual pixels
- This string serves as an identifier, a kind of fingerprint, as it is unique to each image,
and is identifiable in copies of that image, even if they’ve been altered to some degree
- It is more than software – the detection of child pornography also requires a particular
arrangement of laws, institutions, technologies, and collaborations to make it work - PhotoDNA is still just the most sophisticated version of blacklist tools like the rudimentary
word filters from a decade before à it can only identify already known child pornography
* The dream of AI content moderation is to be able to detect not just copies of known
objectionable content and their close variants, but new instances of objectionable content
as well à to replace the work of human moderators
* Computer scientists are now looking to machine-learning techniques, which use large,
existing datasets to train an algorithm to identify qualities within that data that might help it
learn to discern different types of conten
- Machine-learning techniques are founded on two unresolvable paradoxes specific to an
algorithmic approach:
- Machine-learning recognition techniques attempt to make a meaningful distinction
without understanding meaning
- These techniques, while they cannot know what a particular post means, what its
author intended, or what effect it had on its recipient, are intended nevertheless to
classify it as pornographic of harassing or hateful, by evaluating only its visual
information
- Automatic detection can assess only what it can know à what can be represented s
data, limited to the data it has - To develop an algorithm to identify objectionable material automatically, you need to
train it on data that has already been identified
- Machine learning depends on starting with a known database, a ‘gold standard;
collection of examples, an agreed upon ‘ground truth’ that becomes the basis upon
which an algorithm can be expected to learn distinctive features
