Multisensory integration

Question 1

Sensory dominance definition and how it is tested

Answer 1

1. Relative weighting of information from different sensory modalities studied using situations of intersensory conflict.
2. Artificially conflicting stimuli presented at suprathreshold levels (primarily in humans).

Question 2

Subthreshold summation definition and how it is tested

Answer 2

1. Supra-additivity and sub-additivity: rules of multisensory integration of information from different modalities.
2. Matching stimuli presented at near-threshold (sub-threshold) levels (primarily in animals).

Question 3

Deep layers of superior colliculus

Answer 3

Deeper layers of SC seem to contain a map of auditory, visual, and somatosensory (touch) space.

Question 4

Stein and Meredith (1993) on SC neurons

Answer 4

At least 3 senses seem to combine on individual neurons (trimodal).

Neurons demonstrate superadditivity. Audition or touch alone does not demonstrate a large response. When weak inputs presented together, the neuron’s response is greater than the sum of the neuron’s response to each input individually (+1207%).

They also found subadditive neurons when visual and auditory inputs were not aligned (temporal desynchronisation).

Question 5

Law of inverse effectiveness

Answer 5

Greatest superadditivity wheb individual inputs are weakest.

Question 6

Spatial coincidence for when visual and auditory stimuli

Answer 6

Response enhancement (superadditivity) if they are presented in the same RF (cues align).

Subadditivity when the auditory cue is taken outside of the neuron’s RF but the visual cue remains inside its RF.

Question 7

Temporal synchrony

Answer 7

Cells respond subadditively when audition arrives before vision. The largest response occurs when visual cue comes slightly before auditory (maybe because vision takes longer to process?).

Question 8

Stein et al. (1989) multisensory integration in an awake cat collecting food in a semi-circular array with 7 locations

Answer 8

Behaviour: a greater perceptage of correct orientating when visual and auditory cues occur in the same location (especially for peripheral locations) but a subadditive response when cues are spatially incongruent (performance worse than just vision alone).

[but what if auditory cue just took attention away? what if the cat thought that auditory cue also signalled reward?]

Question 9

Stein et al. (1996) human psychophysical task with multisensory integration

Answer 9

Semi-circular setup. Participants had to indicate how bright the target light was by turning a nob.

A task-irrelevant sound seemed to make the light brighter. Results fit with law of inverse effectiveness (greater effect for faint lights).

Question 10

Stein et al. (1996) 2 follow up studies for human psychophysical task on multisensory integration with light and sound cue.

Answer 10

1. Spatial incongruence between vision and audition did not lead to subadditivity (just having a sound increased brightness regardless).
2. Superadditive interactions occurred only for visual stimuli at fixation (not periphery).

Question 11

Odgaard et al.’s (2003) intepretation of Stein et al. (1996) human multisensory integration task.

Answer 11

The sound introduced a response bias, rather than the light actually appearing brighter. (Boost when VA co-occur 50% of the time but not 25%).

Question 12

Risberg and Lubker (1978)

Answer 12

Improvement of correctly perceived speech when we can read lips alongside lowpass-filtered speech.

Question 13

Calvert et al. (2000) fMRI scanner while listening or watching someone read Orwell’s 1984

Answer 13

Left superior temporal sulcus. Increased firing for congruent VA (superadditivity), and subadditive response when VA info desynchronised.

Question 14

Beauchamp et al.’s (2004) argument against Calvert et al.’s (2000) Orwell fMRI study

Answer 14

Spatial resolution of fMRI may be too low to capture the patchy organisation of the STS. It may not be super-additive responses if visual and auditory areas are adjacent and only appear to boost activity when they combine.

Question 15

Senkowski et al. (2011) in human multisensory integration

Answer 15

Multisensory interactions in early evoked
brain activity follow the principle of
inverse effectiveness

Question 16

Spence et al. (2013) on the spatial rule

Answer 16

Not as well-supported as many believe. It depends on the task.

Question 17

Proportion of cells showing multisensory integration

Answer 17

21% are subadditive and 15% are superadditive (only a subset of neurons). Surely if they were put together (animal behaviour controlled by overall average response), we would just get normal additivity?

Question 18

Multisensory integration in awake cats

Answer 18

There seem to be inconsistent results.

Question 19

Holmes’ (2009) argument against multisensory integration

Answer 19

There is nothing magical about superadditivity at all. Maybe it’s just a statistical artefact rather than genuine neural integration.

Question 20

Wuerger et al. (2003) on MI of motion information

Answer 20

MI can’t account for MI of motion information.

Question 21

Guest & Spence (2003) MI of multisensory texture perception

Answer 21

MI can’t account for multisensory texture perception

Question 22

Ma et al. (2009) on speech integration (Bayesian model)

Answer 22

Found an inverted-U shape for enhancement (the most multisensory integration for intermediate stimuli). Contrasts law of inverse effectiveness and fits more with a Bayes-optimal pattern.

Question 23

Driver & Noesselt (2008) on multisensory areas in the brain

Answer 23

There is a multitude of multisensory areas in the brain.

Question 24

Superadditivity and subadditivity in single cells and in behaviour.

Answer 24

1. Superadditivity and subadditivity common at level of single cell.
2. Only some evidence that this extends to behaviour.