lecture 19 - cross modal processing

Question 1

Fodor 1983

Answer 1

proposed that cognition is a collection of independent information-processing modules

Question 2

cross-modal processing

Answer 2

there is evidence that modules can change specificity, may be correlated in some tasks and can communicate with each other

Question 3

von Melchner et al 2000

Answer 3

reported that cross-modal projection in the auditory cortex can mediate visual behaviour = when light stimuli are presented in the portion of the visual field that is “seen” only by this projection, “rewired” ferrets respond as though they perceive the stimuli to be visual rather than auditory

Question 4

modules can be correlated

Answer 4

one example is speech processing where visual and audio information are processed together. in a noisy environment, understanding someones speech can be facilitated by reading their lip movement

Question 5

Spence 2002

Answer 5

demonstrated that visual cues can improve speech perception by up to 15-20 dB. this facilitation is possible because visual and auditory cues are normally correlated - a particular sound will be accompanied by a specific mouth shape

Question 6

McGurk effect

Answer 6

when an audio stimulis is B, but the visual stimulus is G, the actual perception becomes D. participants shut their eyes, they can hear B, but when they see the incongruent G lip movement, they hear D. this demonstrates that neither visual or audio takes precedence and both streams of information work together to allow a new coherent perception

Question 7

multi-modal processing - there are multiple examples of multi-modal processing

Answer 7

cross-modal cuing, modality dominance, one sense enhancing another

Question 8

cross-modal cuing

Answer 8

two pathways of information channels is better than one.

Question 9

Butter et al 1989; Spence et al 2009

Answer 9

when participants are required to detect a faint visual target on the left or right fixation, if they are also given and additional auditory or tactile cue on the side of the stimuli, reaction times are faster

Question 10

McDonald et al 2000

Answer 10

asked participants to detect a weak visual stimulus that was presented to the left or right of a fixation point. the visual target was preceded by an audio cue, either on the same side of the visual target (valid) or on the other side (invalid). 2 variables were measures (low level = perceptual detectability measured as d’ and high level = decision criterion called bias or B)

Question 11

what did McDonald et al find

Answer 11

that d’ was significantly higher when audio was presented on the same side, but there were inconsistent effects on bias B

Question 12

what did the results of McDonald indicate

Answer 12

the results indicated that cross-modal cuing does involve some modulation of relatively low-level sensory processing, but there was no evidence of high-level involvement in this study

Question 13

Dominance

Answer 13

relates to which sesnse is prioritiesed when there is conflicting information. accordingly, which type of sensory dominance (visual or auditory) emerges may be dependent on the conditions with which the stimuli are presented

Question 14

Colavita effect - Covalita 1974

Answer 14

presented suprathreshold auditory and visual targets to participants and asked them to press one button for auditory targets and another for visual targets. one some trials, both the light and sound were presented at the same time/ these trials were interleved into the sequence of unimodal target trials

Question 15

what were the results of Covalita 1974

Answer 15

results found that most participants percieved the visual on the light-sound trials. some reported being unaware of the sound component. this demonstrates visual dominance over auditory stimuli

Question 16

modality appropriateness hypothesis

Answer 16

certain properties of stimuli are better processed by a specific sense, leasing to a particular modality dominating

Question 17

Posner et al 1976

Answer 17

visual modality is superior at processing spatial information

Question 18

Shams, Kamitani and Shimojo 2002

Answer 18

auditory modality is superior for temporal information

Question 19

biased/integrated competition hypothesis - Spence, Parise and Chen 2012

Answer 19

suggest that brain systems dedicated to the different sensory systems may compete. they suggest that given a large proportion of the brain is dedicated to visual processing, visual stimuli and the visual system should be more likely to dominate and inhibit processing in other sensory systems

Question 20

visual dominance

Answer 20

this is when the apparent location of a sensory stimulus can be influenced by visual information. during a venreiloquists act, the mouth of the puppet does not correspond to the location of the sound source, yet observers perceive the puppets speech come from its mouth

Question 21

Bitvinick and Cohern 1998

Answer 21

visual dominance in the rubber hand illusion. the experiementer had to stroke a rubber hand of the participant while the participant was looking at the hand. the participant felt the stroking sensation

Question 22

Shams et al 2000

Answer 22

presented a visual illusion that is induced by sound by sound. when a single flash is accompanied by two bursts of noise, you may perceive two flashes instead of one

Question 23

what did Shams study show

Answer 23

showed how visual stimulus can be altered by another modality, even when the visual stimulus is not ambiguous. this demonstrates that although vision is usually very dominant, like other modalities, it is also malleable by other senses

Question 24

one sense enhancing another - smell and colour

Answer 24

the strawberry smell of a liquid appears stronger when the liquid is coloured red (Sellner and Kautz 1990)

Question 25

one sense enhancing another - sound and light

Answer 25

auditory noise presented with light tends to be perceived as louder than nouse presented alone (Odgaard 2004)

Question 26

one sense enhancing another - light and touch

Answer 26

tactile discrimination thresholds are lower during visual observation (Kennett et al 2001)

Question 27

one sense enhancing another - sound and touch

Answer 27

pertubing the sound made as hands are rubbed together can alter the perception of the skins texture (Jousmaki and Hari 1998; Guest et al 2002)

Question 28

neural basis to multimodal processing - convergence zones

Answer 28

convergence zones in the brain have been suggested for multimodal processing. these are higher-level areas such as the superior temporal sulcus, the parietal cortex, the prefrontal cortex and the subcortical nuclei

Question 29

neural basis to multimodal processing - subcortical nuclei

Answer 29

some neurons in the subcortical nuclei show superadditive responses to multimodal stimulation, where stimulation from multiple senses is greater than the sum of response from the difference senses in isolation

Question 30

neural basis to multimodal processing - visual and auditory cortex

Answer 30

Some researchers have also found multimodal responses in previously considered modality-specific areas, such as the visual cortex and auditory cortex.

Question 31

neural basis to multimodal processing - multimodal/polysensory

Answer 31

there is increased evidence that signals can travel laterally from one unimodal area to another, but there is also some downward travel from the higher-order multimodal/polysensory cortex (STP)

Question 32

Bouba-Kiki effect - Ćwiek et al. 2022

Answer 32

cross-cultural phenomenon where 917 participants speaking 25 different languages maintain a consistently in bouba/kiki identification regardless of their native language

Question 33

synaesthesis

Answer 33

multisensory phenomenon in which stimulation of one sense caused sensory experience in another. people who experience this are called synaesthetes

Question 34

one of the most common types of synaesthesia

Answer 34

colour-graoheme association where words/letters/numbers have particular colours

Question 35

Simner 2019

Answer 35

reported 128 different types of synaesthesia and the number of people with synaesthesia amointed to 4.4% of the population

Question 36

lexical-gustatory

Answer 36

words have different tastes

Question 37

sound-colour

Answer 37

sounds have different colours

Question 38

flavour-colour

Answer 38

tastes have different colours

Question 39

touch-colour

Answer 39

textures have different colours

Question 40

visual-auditory

Answer 40

silent moving objects produce sounds

Question 41

important features of synaesthesia - 1

Answer 41

the experiences are elicited by particular stimuli that would not evoke such experiences in most members of the population. triggering stimuli could be perceptual or conceptual

Question 42

important features of synaesthesia - 2

Answer 42

the experiences are automatic and are exptremely difficult to suppress

Question 43

important features of synaesthesia - 3

Answer 43

the nature of the synaesthetic itself is a conscious perceptual event

Question 44

important features of synaesthesia - 4

Answer 44

the synaesthetic experience is still consistent over time, although this is still debated

Question 45

bottom-up factors - Witthoft and Winawer 2006

Answer 45

showed that for some grapheme-colour synesthetes altering the fonts and cases of latters can affect the saturation of stanesthetic colours. in their study, the colour category of the synaesthetic colours were not strongly affected by these manipulations, only the saturation

Question 46

bottom-up factors - Dixon et al 2006

Answer 46

found that the colour of the ambiguous grapheme was determined by the context

Question 47

Howes 2006

Answer 47

suggested that culture organises the senses, which descend via the physiological level of the brain. they suggested that drawing up an inventory of the range of cultural practices and technologies can help us understand different sensory combos and types of synaesthetic across different cultures and historical periods

Question 48

hearing smells

Answer 48

people in Papua-New Guinea and Trobriand Islands talk about hearing smells

Question 49

Stevenson and Boakes 2004

Answer 49

claim that “Odor display taste properties but do not elicit auditory or visual sensations” may be true for western societies but there is evidence of audio-olfactory synasthesia in other cultures.

Question 50

Howes 2003

Answer 50

This audio-olfactory synasthesia may have arisen because in these locations, most communication takes place face-to-face (i.e. within olfactory range of the other) and odoriferous substances (e.g. anointing the body with oil, chewing ginger) are regularly used to augment the power of a person’s presence and words

Question 51

Calame-Griaule 1986

Answer 51

hearing-smell associations are also common in various African Languages, such as Dogon. those who speak Dogon, speech has material properties that despite its invisible nature are more than just sound and odour both arise from vibrations

Question 52

Dogon also classify words by smell

Answer 52

good speech smells sweet, and bad or impetuous speech smells rotten. A Dogon expression states that “the mouth is too ready to speak is likened to the rectum”.