Week 13 Flashcards
(25 cards)
What are features of OpenFace
▶ Comprehensive Features:
▶ Facial Landmark Detection
▶ Head Pose Estimation
▶ Facial Action Unit Recognition
▶ Eye-Gaze Estimation
What is OpenFace
▶ A toolkit developed initially at Carnegie Mellon University
▶ Provides real-time analysis of facial behavior
▶ Non-invasive; real-time data
What is head pose estimation
▶ Measures tilt, rotation, nodding to interpret conversational cues
▶ Application in Communication Studies:
▶ Enhances understanding of non-verbal cues in dialogues
What is facial action unit recognition
▶ Identifies and quantifies muscle movements for emotional analysis
▶ Provides insights into emotional underpinnings of speech
What is eye-gaze estimation
▶ Provides data on where and what people focus on during interactions
▶ Crucial for understanding how infants associate visual cues with verbal information
What is a context-sensitive model for speech coordination
▶ Sensitive to upcoming context.
▶ Movements are pre-planned to anticipate future articulations.
▶ Key question: How do my articulators usually coordinate the upcoming sequence?
▶ Use of motor plans: stored motion patterns.
▶ Built from:
▶ Motor primitives: smallest planning units (e.g., a single articulator movement).
▶ Motor programs: combinations of motor primitives.
▶ Motor plans: larger structures (phrases, sentences)
What are context-invariant models in speech coordination
▶ No “look-aheads”
▶ Articulators average together adjacent movements automatically.
▶ Key question: What’s the fastest way to get articulators from here to there?
▶ Example: Task Dynamics approach
What is the motor planning and task dynamics unifying theory for the coordination problem?
Formal theories propose a combination of the two
▶ They act in two different levels:
▶ Motor program level :
▶ Stores abstract, reusable motor patterns.
▶ Guides the selection and sequencing of articulatory targets.
▶ Responsible for higher-level planning (e.g., syllables, words, phrases).
▶ Motor system level :
▶ Handles real-time execution of motor plans.
▶ Implements the physical dynamics needed to reach specified articulatory targets.
▶ Resolves biomechanical constraints and fine-tunes motion paths.
▶ Together, these levels allow for both pre-planned structure and real-time
adaptability in speech
What are complex sounds
▶ Sounds that involve simultaneous use of multiple articulators
▶ Examples: liquids, clicks, nasalized and voiced sounds.
What is the Task Dynamics approach
Task Dynamics
▶ Define a task as movement toward a physical target.
▶ Dynamics: calculate forces needed to get from current to target state.
▶ No need for stored plan; things are computed on the fly.
▶ Pseudoinverse solutions determine efficient paths
What is the inverse problem in speech
The Inverse Problem in Speech
▶ Definition: Inferring the cause from an observed effect.
▶ In speech research: Given an acoustic signal, determine the articulatory movements
that produced it.
▶ Challenge: Multiple articulatory configurations can yield similar acoustic outputs.
▶ Example:
▶ The vowel [i] can be produced with slightly different tongue and lip positions, but
still sound the same
What is the pseudoinverse solution in speech modelling
Definition: A mathematical way to find the best possible solution to an inverse
problem when no exact solution exists.
▶ In speech research: Used in task dynamics to compute optimal articulator
trajectories to reach a goal.
▶ Why pseudoinverse?
▶ The vocal tract is redundant — multiple ways to achieve the same sound.
▶ The pseudoinverse finds the most efficient combination of articulatory commands
What is the timing problem
▶ Many speech sounds involve multiple articulatory movements.
▶ Question: How is timing between movements controlled?
▶ One idea: a central “clock” regulates movement timing.
▶ Known as extrinsic timing.
▶ Problem: No strong evidence for such timing device
How does task dynamics offer a solution to the timing problem
▶ Task Dynamics offers an alternative: intrinsic timing.
▶ Relative movement cycles rather than absolute time.
▶ Phase alignment = timing relationship between gestures.
▶ Languages tend to show recurring global phase patterns
what is anatomical coupling
▶ The human vocal tract is a connected system of articulators.
▶ Some articulators move together due to physical or muscular linkages.
▶ Anatomical coupling refers to the degree to which two articulators influence each
other’s movement.
▶ Tightly coupled articulators: movements in one often affect the other.
▶ Independent articulators: can move without significantly affecting each other
Give examples of tightly coupled articulators
▶ Tongue tip and tongue body — shared musculature leads to interdependence.
▶ Lips and jaw — jaw lowering often affects lip position.
▶ Example from the book: A three-legged race is difficult b/c of reduced
independence
Give examples of relatively independent articulators
▶ Velum — moves independently from most oral articulators.
▶ Larynx — voicing and glottal gestures can co-occur with almost any other
articulation
Why are anatomically independent articulators important
▶ Ease of Coordination: Independent articulators are easier to coordinate → more
combinations are possible.
▶ Typological Trends: Sounds that combine independent articulators (e.g., nasal +
voiced) are more common across languages.
▶ Learning Difficulty: Sounds requiring tightly coupled articulators (e.g., multiple
lingual gestures) are harder to learn, especially for second-language learners
What are lingual-lingual sounds
▶ Tongue = jointless hydrostat; complex muscle interactions.
▶ Lingual-lingual sounds require two distinct tongue constrictions.
▶ Due to tight coupling, these sounds are:
▶ Hard to learn.
▶ Relatively rare cross-linguistically.
▶ Focus: Liquids and Clicks
Do you think dark /~l/ is harder to learn than light (clear) /l/?
A. Yes
B. No
A
Why is dark l rare in languages
Involves more coordination of anterior and posterior tongue gestures, which are
anatomically coupled
What is articulatory overlap
Overlap = simultaneous movement of different articulators
Common when articulators aren’t anatomically coupled
What is articulatory conflict
when adjacent speech sounds require anatomically coupled articulators to move in
opposite directions, we get a conflict
What are ways to resolve articulatory conflict
▶ Deletion: we can delete one of them
▶ Transition: we move the articulator from one position to the other
▶ Compromise: we pick a middle ground between them
