Multisensory Integration Flashcards Preview

PSY2002 Cognitive > Multisensory Integration > Flashcards

Flashcards in Multisensory Integration Deck (41)
Loading flashcards...

Perception to action

Psychophysics- present sensory stimuli and report if they felt anything

Lighting a match- seemingly easy, reaching, grasping, manipulating objects

Seemingly hard logic puzzles

Computers are good at hard tasks

Reach/grasps- robots are bad at seemingly easy tasks


Moravec’s paradox


High level cognitive reasoning tasks- easy for robots

Low level cognitive tasks, perception- hard

Computers show adult level performance on intelligence test/ playing checkers

To give them skills of 1yrs old concerning perception and mobility

Bias- we assumed if these tasks were easy to us they would be simple for robot


Lighting match

Anaesthetise finger- loss of touch sensation from fingers - sight still there

Does not affect motor control

When sensation is blocked, much harder to light the match

25s- block sensation
5s- normal

Shows vision and sensation very important for simple task


Multi sensory integration

Touch and vision integrate


Challenges in multi sensory integration

Transforming representation from different senses to common representational space

Integrating info from different senses into coherent percept -> mismatch


Reference frame

Representation schemas of information from different schemas

Snake game:
Player perspective: coordinates of snake in the game-top-down view

Snake perspective: see the world through its eyes- turn left/ right

Problem: sensory input in player, control in snake-> need to know info about body layout of snake


Reference frames for different senses

Vision = eye centred/ retinal- location of visual stimulus on retina

Audition= head centred- location of sound source in respect to ears

Touch= body centred- location of tactile stimulus on skin

Need to convert between the reference frames and to external space


Coordinate transformation

Dog barking and seeing dog= separate reference frames

How to convert between two and to know the difference between the two you have to know

Angle between two

Converting between frames- have to know position and orientation of body parts


The body schema

Spatial coded: position of each body part in external space

Modular: different body parts processed in different brain regions

Updated with movement: automated and continuous tracking of body posture

Adaptable: changes when body changes

Supramodel: combines input from proprioception, touch and vision

Coherent: resolves perceptual conflicts

Interpersonal: observed actions are represented within the same body schema


Types of body representation

Body schema: sensorimotor representation that guides action

Body image: body percept, body concept, body affect -> how we think/ feel about our current body


Doe Body posture affect perception

Temporal judgement task

Stimulate both hands in random order, pots have to stretch fingers of hand stimulated

Arms crossed / arms uncrossed

When arms crossed- ppts mix up which hand was stimulated

Solving task- do not need input from body schema when arms uncrossed

Body schema interferes with basic perception


How does a body schema develop

6mnth old
Schema starts to interfere with tactile orienting
Shown when crossing/uncrossing feet and buzzing one

4mnths- no difference I’d crossed, reach for right foot

6mnth- baby reaches for correct foot, more correct when feet uncrossed


No arm crossing

Tactile discrimination task- determine which finger was vibrated

Visual distract either on same hand or other hand

Distractors led to response delays

Congruent distracts lead to longer delays than incongruent

62ms vs 20ms


Arm crossing

Tactile tumulus in same side of body, visual stimulus on different side

Effect of visual distractor moves with the hand during arm crossing

Cross modal interactions mediated by body schema

Opposite response

67ms vs 3ms


Peri personal space (PPS)

Space immediately surrounding our bodies

Objects in PPS can be grasped and manipulated immediately


Tool use

Extending the body

Tools are incorporated into the body schema

Cross modal congruency effects apply during tool use

No crossing of body parts only tools= same delay effects

Tools become part of the body schema, represented same in brain


Alice in wonderland syndrome

Distortion in perception of size

Body parts might appear smaller (micromatognosia) or bigger than they are (macrosomatognosia)

Affects whole body

Associated with childhood and migraines



Unable to locate body parts

Loss of spatial unity of body

Patients can name body parts but order is lost
Can’t point out where body part is, unaware of how it looks

Finger agnosia- fused percept of finger, can not indicate what finger was stimulated


Phantom limb

Still feel presence of limb even after loss of limb

May include agency/ movement

Associated with pain

Can change size over time- shrink / telescoping


Double dissociation

Causes decomposition or concept of body representation


Cross modal neurons

Neuron fires when you put an object within range/ touches of the hand- respond when seen is touched / object moves near hand

Neuron responds to visual and tactile stimuli

Visual receptive friend moves with the position of the hand

Modified by body schema


Neurons incorporating tools

When monkey holding tool response space expands

Expansion of Peripersonal space during tool use reflected in neural response

As response field expands, body schema encompasses the tool


Integration problem

To see something- represented in external space

To hear- auditory input- represented in a different reference frame

Have to view according to both these inputs -> sensory conflict


Sensory conflict

Different senses might provide conflicting information about a sensory stimulus

Needs to be resolved


Testing conflicts between vision and touch

Judge size of object by vision and touch

Look through reducing lens

If we follow vision object looks a lot smaller

If we go by touch object feels a lot bigger


Visual capture

Assess the size of a cube via pointing, feeling, drawing

Vision dominates perceived object size- visual capture- trust visual sense much more


Sensory hierarchy?

Auditory can dominate vision

I report no. of visual flashes seen

Auditory beeps played during flashes

No. of auditory beeps determines reported no. of visual flashes


Modality precision hypothesis

Modality with highest precision (lowest uncertainty) is chosen dependent on task

Spatial task- choose vision as highest accuracy

Temporal task- audition much higher accuracy


Sensory uncertainty occurs due to

Perceptual limits- visual resolution determined by spacing of photoreceptors in fovea

Neural noise- synaptic noise

Cognitive resource limits- attention


Sensory modality changes

Emst and banks (2002)

Created artificial conflict- judge height of bar

Visual height and haptic height

Virtuality reality set up- bar not there- illusion

Change height of bar, modify uncertainty by adding visual noise

Haptic- force feedback device, changes height of bar, judging size of bar by touch

Vision and haptic input can be conflicting