Adaption

Question 1

Adaptation

Answer 1

• describes the process by which the visual system alters its operating properties in response to recent changes in the environment
• widespread property of neural sensory systems
• effects range from influencing response of individual cells in the visual brain to shaping our perception of the world

always happening, passive process

Question 2

luminance of a retinal image depends on 2 factors

Answer 2

1. surface illumination
2. relative surface reflectance

Question 3

surface illumination

Answer 3

the amount of light falling onto an object

Question 4

relative surface reflectance

(albedo)

Answer 4

the proportion of light that is reflected back from the surface

how much light is absorbed & how much is reflected to our eyes

Question 5

dynamic range of light intensity

Answer 5

the luminance of a pieve of white paper is 1b times higher in outdoor sunlight than on a moonless night

Question 6

output range of the retina

Answer 6

• the output cells of the retina (RGC) have a limited response range - ~1-300 spikes per second
• the retina must try to:
• accomodate large changes in mean luminance
• maintain sensitivity to differences in luminance within a scene

Question 7

pupil changes

Answer 7

• the overall amount of light entering the eye is regulated by controlling the size of the circular pupil
• pupillary light reflex
• but in humans this accounts for only a 10-fold to 20-fold change in the intensity of light on the retina - still 8 orders of magnitude
• vertical slit pupils found in other species are slightly more effective e.g. cats

Question 8

pupillary light reflex

Answer 8

high intensity light causes the pupil causes the pupil to become smaller, whereas low intensity light causes the pupil to become larger

lets us change how much light comes in

Question 9

duplex function

Answer 9

• the operating range of the retina is increased by having distinct receptor systems which are specialised for different lighting conditions
• scotopic vision = low light conditions, rods
• photopic vision = well lit conditions, cones

overlap = mesopic vision, use both rods and cones

Question 10

rods

duplex function

Answer 10

• highly sensitive (can respond to a single photon of light)
• high convergence (average of 120 rods to one RGC)
• saturate under daylight conditions

Question 11

cones

duplex function

Answer 11

• less light sensitive (requires 100s of photons to respond)
• low convergence (average of 6 cones to one RGC, but one-to-one relationship in fovea)
• continue to respond in high light conditions

Question 12

dark adaptation

Answer 12

• what happens if we suddenly shift from a well lit area to a darkened room?
• at first it is difficult to see anything, but over time you adjust to the lower light conditions around it becomes easier to see your way around

Question 13

measuring dark adaptation

Answer 13

• expose subject to a uniform adapting light source
• sit them in a darkened room
• flash a very dim light with increasing intensity until the subject can just detect it
• repeat this process at regular time intervals
• plot the subject’s light detection threshold as a function of time in the dark

trying to find threshold sensitivity

Question 14

viewing conditions for a dark adaptation experiment

Answer 14

the image of the fixation point falls on the fovea and the image of the test light falls in the peripheral retina

• present off to the side
• stimulate the part of the retina that has both rods & cone photoreceptors

Question 15

key features of dark adaptation

Answer 15

• increase in sensitivity over time
• two distinct branches, due to the transition from cones to rods
• within each branch there is significant adaptation
• cone sensitivity adjustment takes 4-5 mins
• rod sensitivity adjustment takes 20-30 mins

Question 16

bleaching/regeneration of photopigment

Answer 16

• after a photopigment molecule is bleached (used to detect a photon of light), the molecule must be regenerated before it can be used again
• in low light levels, more rod and cone photopigments are gradually regenerated, increasing their ability to respond to photons of light
• recovery of rod photopigment (rhodopsin) is slower than the 3 types of cone photopigments

• concentration of photopigments (how many are available at a given time)
• rhodopsin = rod photopigment
• bleaching - retinal and opsin break apart
• how it turns into neural response
• increasing time = greater concentration which means we are more sensitive to light

Question 17

negative feedback

Answer 17

• feedback from horizontal cells onto the photoreceptors also changes their sensitivity
• if horizontal cells respond strongly they tell the photoreceptors to turn it down

• horizontal cells give inhibitory feedback
• in-built sensitivity check with retina

Question 18

light adaptation

Answer 18

• incoming light bleaches the plentiful rod and cone photopigments, leading to a massive firing in the RGCs
• at first everything seems glaringly bright and ‘washed out’, but things return to normal after a minute or so
• reverse to dark adaptation: pupil constricts to reduce light input, shift to photopic system, reduction in photoreceptor sensitivity

Question 19

light and dark adaptation in real world

Answer 19

• why some fighter pilots wear red glasses before a night flight - rods are not sensitive to red light, making it possible to dark adapt prior to entering dark conditions
• why star-gazers use red torches - avoid bleaching rod system once dark adapted

red goggles block out all light except long wavelengths

Question 20

perceptual consequences of light/dark adaptation

lightness constancy

Answer 20

• white paper inside a room reflects less light than black paper does outside
• perception of lightness doesnt simply reflect the absolute luminance of an object
• luminance constrast - the difference between an object’s luminance and the background
• in the photopic range, the entire operating range of RGCs shifts according to mean light level
• known as gain control & results in responses that represent contrast rather than absolute intensity
• gain control in retina ensures that the visual system maintains sensitivity to contrast despite large changes in mean luminance
• weber’s law & just discriminable difference

• when changing the sensitivity of the retina, proportions will stay the same
• respond to percentage rather than absolute light level
• gain control: matching output to input

Question 21

consequences of light/dark adaptation

negative afterimages

Answer 21

• complementary coloured aftereffects occur due to the opponent nature of chromatic pathways
• e.g. adaptation to red causes a reduction in the sensitvity of long wavelength cones, creating an imbalance in the inputs to red/green opponent RGCs
• Van Lier et al. (2009) - some form of ‘filling-in- of the afterimage across shape contour

• byproduct of shift in sensitivity
• using colour opponency and sensitivities in retina

Question 22

perceptual consequences of light/dark adaptation

troxler fading

Answer 22

• under steady fixation, stationary objects in the periphery tend to fade and disappear
• movement of the eyes immediately restores visibility
• torxler fading only occurs in peripheral vision, most likely because receptive fields in the periphery are considerably larger than our fixational eye movements

• faint stimulus in periphery
• lilac chase = troxler fading and negative afterimaging

Question 23

V1 neural response properties

Answer 23

• small receptive fields
• orientation and spatial frequency tuning
• contrast-dependent response
• binocularity

• small receptive fields: Only respond to visual input from small area of space. excitatory & inhibitory regions
• areas are elongated
• contrast-dependent response. diffs between darkest & lightest regions in space
• binocularity: information from two eyes get inputted into one another

Question 24

contrast adaptation in V1

Answer 24

repeated presentation of a preferred stimulus reduces neural responsivity

• over time: reduction in responsivity
• how well they responded as a function of the contrasts
• bursting pattern corresponding to graphs
• reduce contrast: less and less response

Question 25

neural fatigue or contrast gain control? | contast adaptation in V1

Answer 25

* following adaptation some V1 neurons show an overall reduction in firing rate, akin to a form of neural fatigue * however many exhibit a lateral shift in their contrast response function, shifting the response range towards the adapted contrast (contrast gain control) ## Footnote * adaptation must be something like fatigue * but some change in a different way. reduction but not scaled down - more like a shift of response function * mixture between 2 patterns

Question 26

selectivity | contrast adaptation in V1

Answer 26

* changes in responsivity depend on the response of a given neuron to the adapting stimulus * as a result contrast adaptation shows similar properties to the response profiles of individual neurons ## Footnote * how much neuron adapts is proportional to how it responds in first place * if adapt to preferred stimulus - big change in response

Question 27

perceptual consequences - contrast sensitivity

Answer 27

* prolonged exposure to a given visual stimulus selectively alters contrast sensitivity * ability to detect patterns similar to the adapting stimulus is impaired * no change in ability to detect different patterns ## Footnote * adapt people to high contrast grating * did grating appear 1st or 2nd interval * lowest contrast when they can detect? * by flickering - don’t get the build-up of afterimage * mimics what we see in the neurons

Question 28

mirroring the response properties of individual neurons in V1, adaptation-induced changes in contrast sensitivity exhibit: | perceptual consequences - contrast sensitivity

Answer 28

* location specificity: adapting at one ocation doesnt alter sensitivity to stimuli presented at a distant location * orientation tuning: elevation in contrast threshold falls off as the difference between adapted and tested orientations is increased * spatial frequency tuning: elevation in contrast threshold falls of as the difference between adapted and tested spatial frequency is increased * partial inter-ocular transfer: adapting to stimuli presented to one eye will affect contrast sensitivity measured in the other eye

Question 29

perceptual consequences - aftereffects

Answer 29

* adaptation to a particular orientation produces a repulsive bias in the perception of nearby orientation (tilt-aftereffect) * adaptation to a given spatial frequency produces repulsive biases in the perception of nearby spatial frequencies (spatial frequency aftereffect) * population coding model of repulsive aftereffects: selective reducation in neural responsivity shifts the neural population response away from the adapted value ## Footnote * contrasts well above our thresholds (supra-threshold) * tuned to vertical but also some to different orientations * explain with simple models

Question 30

function of contrast adaptation: improve performance | contrast gain control

Answer 30

* adaptation shifts steepest part of contrast response function towards prevailing contrast level * should allow observers to better discriminate differences in contrast around this level * selective improvement in contrast discrimination has been found in some psychophysical studies, but not others ## Footnote * shift in contrast response along contrast axis * steepest part line up with adapting contrast. should make better in picking up differences. steepest = small change in response * get better at picking up small changes * when test - not that convincing, little studies find this * repeat for different pedestal increments * adaptation functioning to improve detecting the contrasts

Question 31

function of contrast adaptation: improve efficiency?

Answer 31

* contrast adaptation acts to improve the efficiency of neural representations * adaptation equalises the response level across populations of neurons over time * this reduces the metabolic cost of continued responses to regular features of the environment ## Footnote * not having neurons respond to things that aren't changing throughout the world * adaptation helps to ensure that this happens * biased distribution. 1 orientation at 0 degrees (vertical) is shown more frequently * can explain responses by fitting tuning functions * on average - vertical orientated didn’t respond any more than other ones * change in response might be just enough that this stays true * adaptation doesn’t make it any better

Question 32

adaptation: the psychophysicist's electrode

Answer 32

* neural adaptation has been documented in the majority of visual cortical areas * psychophysicists often use adaptation as a tool to learn more about visual processing at different levels of the visual hierarchy ## Footnote as we move up hierarchy of processing: receptive fields get bigger, progressive change in selectivity

Question 33

adaptation to subjective contours

Answer 33

* subjective or illusory contours are lines or edges perceived where there is no luminance or colour difference * neurons in V1 respond only to real contours * 40% of neurons in V2 respond to subjective contours * adaptation to subjective contours induces tilt aftereffects which closely resemble those produced obtained by luminance contours

Question 34

adaptation to shape

Answer 34

* Suzuki (2005) - demonstrated aftereffects following adaptation to a variety of shape properties, such as convexity and aspect ratio * adaptation to shape ot just multiple local tilt aftereffects? * some evidence of distinctive tuning properties: * some tolerance to changes in size/position, occur for very brief adaptation periods, broad orientation tuning

Question 35

adaptation to global form

Answer 35

* adaptation to glass patterns containing concentric or radial structure results in the perception of structure in completely random test patterns * local or global adaptation? * similar magnitude aftereffects found with simple linear structure * manipulation of the position of adapting and test patterns revealed weak position invariance * likely invovles combination of adaptation at multiple levels of form processing ## Footnote * dot stimuli made up of pairs of dots (dipoles) * consistent with concentric form (circular form) - top left circle * if adapt to concentric then presented with random

Question 36

adaptation to faces

Answer 36

* adaptation to a distorted face makes the original face look distorted in the opposite direction * face morphing approaches have been used to investigate a range of aftereffects ## Footnote * range of aftereffects consistent with repulsion idea * take whole set of pictures - morph them together to get neutral face * adapting to female makes neutral face look more male

Question 37

properties of face aftereffects

Answer 37

* evidence suggests that face distortion effects reflect a mixture of low level and high level adaptation processes e.g. * size tolerance: manipulating the relative size of adapting and test reduces, but doesnt abolish face distortion aftereffects altogether * orientation tolerance: inverting the test face reduces the magnitude of face distortion aftereffects * object specificity: adaptation to distorted faces altered faces alters the appearance of other objects, though not to the same extent as other faces

Question 38

multidimensional face space | norm-based coding

Answer 38

* it has been proposed that the visual brain represents faces in a fundamentally different way to many basic image properties * in a norm-based representation, each individual face is coded by how it deviates from a prototype or average face on multiple dimensions * morphs between a given face and the norm lie on an identity axis, with distinctiveness increasing with distance from the centre * 'anti-faces' can be produced by extrapolating the identity axis beyond the norm ## Footnote * suggestion that they are represented in the brain in a fundamentally different way to other things (faces) * faces - coding happens in a different way * prototypical face = average of others. every other face as some deviation of that. further you are from middle - easier to distinguish

Question 39

adaptation to faces | identity aftereffects

Answer 39

* adaptation to an anti-face makes average face look like original face * recognition of original face improved, but recognition of other identities impaired * consistent with the notion that adaptation shifts the norm in face space towards the adapted face ## Footnote * adapt to anti-henry makes average look more like henry * average gets pushed out towards original * harder to detect anti-face

Question 40

function of face adaptation: normalisation?

Answer 40

* normalisation accounnts for properties of face adaptation that are at odds with traditional repulsive aftereffects * adaptation changes the appearance of the adapted face, making it look less distinctive * adaptation to the neutral face doesn't alter the appearance of a distorted face

Question 41

what is motion good for?

Answer 41

* motion-based image segmentation * navigation * depth from motion * structure from motion ## Footnote * segmentation: in static form, hard to see what is happening. When moving easier. motion good cue for grouping objects together * depth: speed at which objects move tells us something about how far away they are * structure: 3dimensional structure, biological motions

Question 42

how is motion computed in the visual brain?

Answer 42

* indirect method - aka feature tracking/cognitive strategy/long-range motion * direct method

Question 43

indirect method | how is motion computed in the visual brain?

Answer 43

* independent analysis of spatial displacements and temporal intervals * used when there are few objects/features, long intervals and/or large displacements (e.g. Braddick, 1974) ## Footnote fundamentally motion is location over time. works well when clearly defined features

Question 44

direct method | how is motion computed in the visual brain?

Answer 44

* specialised detectors compute motion directly from intensity variations in the retinal image without feature tracking * motion perception remains possible with: * sub-thresholds spatial and temporal displacements (Exner, 1875) * brief, high density displays which preclude cognitive tracking of features (Braddick, 1980) ## Footnote possible to perceive even when distance and time is short

Question 45

Exner (1875)

Answer 45

* placed close together 2 sources of sparks that could be triggered in sequence * noted that observers could see movement between the sources when they were placed so close together that the two sources could not be resolved when triggered at the same time and when the temporal interval was so short that pairs of sparks at the same source could not be distinguished from a single spark * this demonstrates that motion does not require the prior computation of spatial displacement or temporal intervals * motion is a primary sensory quality not a secondary effect

Question 46

motion-sensitive receptive fields

Answer 46

* luminance profiles of moving contours are oriented in space-time * as a result, detecting motion is analogous to extracting spatial orientation * some V1 cells have receptive fields which oriented in space-time - they respond strongly to oriented edges moving in a preferred direction, but not all in the opposite direction ## Footnote * starts to drift to left - end up with 2 dimensional pattern * shows orientation * picking up on change in location over time * V1 - some neurons that have receptive fields that look like this

Question 47

reichardt model of motion detection | motion-sensitive receptive fields

Answer 47

* receptive fields sample two adjacent points in space (neurons A and B) * moving stimulus activates each in turn (rightwards moving bar) * output of one neuron delayed relative to the other (D) * two outputs compared (X) * if internal delay matches time it takes for stimulus to move between receptive fields, detector (M) will signal rightwards motion * movement in opposite (leftwards) direction gives no response ## Footnote * trying to pick up on bar moving left to right * multiple receptive fields that pick up on this well. non-motion-sensitive * to get motion-sensitive: add in extra stage, add in delay to first, both reach common location at coincident time * tuning delays we can get extra information about motion

Question 48

aperture problem | limitations of local motion analysis

Answer 48

* local motion detectors in V1 only 'see' a small part of the image and respond to motion orthogonal to luminance edges * as a result the output of any one motion dector may not be a valid indicator of the overall direction an object is moving * solution: combine the responses of detectors with receptive fields located in different regions of space ## Footnote * analogous to V1 receptor fields * need to take output of lots of neurons in V1 and combine to understand what the actual movement is

Question 49

speed selectivity | limitations of local motion analysis

Answer 49

* speed is the ration of temporal frequency and spatial frequency * local motion detectors in V1 respond to one particular combination of SF & TF, rather than to all stimuli moving at a particular speed * solution: combine the output of dectors that respond to the same TF/SF ratio ## Footnote * speed depends on both temporal and spatial frequency * needs to respond to all combinations. tends not to happen

Question 50

motion integration in MT

Answer 50

* MT neurons have large receptive fields, receiving input from many V1 neurons (ratio of RF dimensions ~ 10:1) * whereas V1 neurons only respond to a particular orientation (edge of bar) moving in a particular direction, MT neurons respond to a preferred direction independent of pattern * many more neurons in MT also show proper speed tuning than in V1 * MT neurons receive input from both eyes and are tuned to different binocular disparities ## Footnote * width of MT is about 10x that of V1 * more global integration in MT

Question 51

higher-order motion processing

Answer 51

* neurons in the medial superior temporal (MST) area have particularly large receptive fields and respond selectively to complex optic flow patterns * neurons in the superior temporal sulcus (STS) respond selectively to biological motion * regions in the intra-parietal sulcs (IPS) and lateral occipital sulcus (LOS) have been implicated in the extraction of structure from motion by fMRI studies ## Footnote * further we go, more complex characteristics * MT/V5 e.g. motion in one direction * MST e.g. expanding motion as drive forward * STS e.g. biological motions * further e.g. 3 dimensional movements

Question 52

primary visual cortex | neural consequences of motion adaptation

Answer 52

* repeated stimulation in the preferred direction reduces the responsivity of direction-selective V1 neurons * as is the case for non-direction selective neurons, adaptation reduces the response of some neurons to stimuli at all contrast levels, but shifts the contrast response function laterally for others ## Footnote * continually stimulate V1 neuron - reduce response over time * function of contrast of stimulus = reduction in sensitivity * shift in function or reduction at all contrast levels

Question 53

MT | neural consequences of motion adaptation

Answer 53

* adaptations to drifting gratings produces a mixture of response gain and contrast gain changes in MT neurons * but, these effects disappear when adapting and test stimuli were presented in different sub-regions of the receptive field * suggests that effects of adaptation are likely inherited from earlier levels e.g. V1 * Priebe et al. (2002) measured adaptation of MT cells over shorter timescale * following the onset of motion, MT neurons display an initial transient peak in firing rate that decays to a sustained level over ~100ms * this reduction in responsiveness is maintained when stimuli are moved to different sub-regions of the receptive field, suggesting that it is not inherited from V1 inputs ## Footnote * ask whether effects seen in MT are happening in MT or inherited from changes in V1 * map out limits * adaptation over fast timescale * typically - initially rapid spike rate that decays over time * even if shifted motion in field, pattern stays the same

Question 54

perceptual consequences of motion adaptation

Answer 54

* motion adaptation produces changes in sensitivity and biases that mirror those seen following adaptation to static stimuli * contrast stimuli - contrast sensitivity is reduced for gratings moving in the adapted direction, but not in the opposite direction * direction aftereffect - adapatation to a given direction of motion produces repulsive biases in the perceived direction of subsequently viewed stimuli ## Footnote * if we adapt to moving stimulus, our ability to locate low contrast stimulus reduced unless it is opposite direction * tuning of effect as manipulate adapted and test direction

Question 55

the motion aftereffect

Answer 55

* after adaptation to a given direction of motion, static or flickering objects appear to move in the opposite direction * aka waterfall illusion

Question 56

distribution shift model (mather, 1980) | explanations of the motion aftereffect

Answer 56

* prior to adaptation, a static test pattern produces similar responses from motion selective neurons tuned to all directions * adaptation selectively reduces neural responsivity to the adapted direction, causing a shift in the distribution of responses to the test pattern ## Footnote * what happens if present a test stimulus: * typically static stimulus - some response but relatively small and matched * response equated for all directions * when everything is balanced - no net motion

Question 57

properties of the static motion aftereffect

Answer 57

* location specificity: adapting at one retinal location doesnt result in a MAE at other locations * spatial frequency tuning: strongest MAE is elicited when the adapting and test stimulus have similar spatial frequency * partial interocular transfer: switching eyes between adapting and test periods reduces the MAE, but doesnt abolish it * temporal frequency tuning: strongest MAE obtained at a constant temporal frequency regardless of stimulus spatial frequency. not tuned to speed

Question 58

the dynamic (flicker) motion aftereffect

Answer 58

* test with a moving or flickering test stimulus and try to null the percept of illusory notion * e.g. stimuli constructed by adding together upwards and downwards drifting gratings with different constrant balances * in the absence of adaptation, equal component contrasts gives rise to the perception of counter-phase flicker * the strength of the MAE can be quantified by measuring the shift in this null point after adaptation in a given direction ## Footnote * using stimuli made up of 2 gratings which drift in opposite directions (middle which doesnt drift) * high contrast in upwards - see middle as upwards * how much the null point shifts

Question 59

properties of the dynamic motion aftereffect

Answer 59

* low positional specificity: adapting at one retinal location induces MAEs at remote spatial locations, particularly when complex optic flow motion used * complete interocular transfer: similar magnitude dynamic MAEs obtained when testing the non-adapted and adapted eye * speed tuning: magnitude of the dynamic MAE is tuned to speed - the ratio of temporal frequency to spatial frequency ## Footnote * no spatial overlap but still strong aftereffects * seems to care more about the speed in dynamic motion aftereffect

Question 60

multiple stages of motion adaptation?

Answer 60

* the differing properties of static and dynamic MAEs has given rise to the suggestion that they may reflect adaptation at local and global stages of motion processing respectively * however questions remained unanswered: * why is it that different types of test pattern reveal different stages of motion adaptation? * do these different MAEs reflect different forms of adaptation seen physiologically? ## Footnote * in both types of aftereffects, adapting stimulus isn't changing, why do you see aftereffects at some? * motion aftereffect seems to work in same way for spatial and temporal

Question 61

evidence for high-level motion adaptation | jordan et al. (2006)

Answer 61

* observers adapted to biological motion stimuli which distinguished males from females * adaptation to the gate of one gender biased judgements of subsequently viewed gates towards the other gender * randomising the phase of each individual dot's significantly reduced the size of the effect, suggesting that it was not due solely to local adaptation

Question 62

improved discriminability? | what is the function of motion adaptation

Answer 62

adaptation has been shown to improve speed discrimination in humans and monkeys | but these effects are modest

Question 63

improved efficiency? | what is the function of motion adaptation

Answer 63

many researchers believe that adaptation improves the efficiency of the neural code by equalises response levels over time and reducing redundancy

Question 64

normalisation? | what is the function of motion adaptation

Answer 64

researchers have argued that motion adaptation acts to recalibrate the zero-velocity point

Question 65

sensory processing: beyond the modular approach

Answer 65

* modular view has been fuelled by findings of distinct cortical areas specialised for processing diff types of sensory input * BUT important to note that there are areas of overlap * by placing sensory cues in conflict, it is possible to demonstrate that our perception is actually shaped by interactions between different sensory modalities

Question 66

the McGurk effect

Answer 66

* sound doesn’t change * different lip movements * because of the different lip movements we hear different things

Question 67

spatial ventrioloquism

Answer 67

* when visual & auditory stim are simultaneously presented at diff spatial locations, the perceived location of the auditory stim often shifted towards the visual location * ventriloquists exploit this to create impression their voice is emanating from dummy's mouth * effect is typically asymmetric - perceived auditory location shifts towards visual location, but perceived visual location largely unaffected by auditory stim ## Footnote sound gets 'pulled' to visual aspect

Question 68

temporal ventriloquism

Answer 68

* direction of interaction is reversed * the perceived timing of visual stimuli biased towards that of asynchronously presented auditory stimuli * visual stimuli have little or no effect on the perceived timing of auditory stimuli ## Footnote * timing of visual events get pulled by auditory events * if one flash but two tones there seems to be two flashes

Question 68

modality appropriateness hypothesis | what rules govern mutlisensory interactions?

Answer 68

* discrepancies between sensory estimates should be resolved in favour of the modality that is most appropriate for the task at hand * visual spatial acuity is typically much better than auditory spatial acuity, so vision 'captures' the spatial location of auditory stimuli * auditory temporal acuity is typically much better than visual spatial acuity, so audition 'captures' visual stimuli in time * BUT recent research has shown that these rules are quite flexible & depend on the balance of unimodal sensitivities for a given judgement ## Footnote * higher temporal resolution for audition than vision * lower spatial resolution for audition than vision * touch tends to be between the two for both types of resolution * use the best modality that is given to you

Question 69

benefits of multisensory integration

Answer 69

* resolves discrepancies associated with internal (neural) and external (environmental) noise, thus helping to maintain a unified percept of the world * increases the precision of perceptual judgements

Question 69

maximum likelihood estimation | what rules govern multisensory interactions?

Answer 69

* provides mathematical framework for understanding integration of sensory cues * proposes that the brain forms a weighted average of the estimates obtained from each sensory modality * large weight is assigned to estimates with low uncertainty, whereas low weight is given to estimates with high uncertainty * this approach has been shown to successfully account for patterns of multisensory integration is a variety of different contexts ## Footnote * predicts over a range of tasks * auditory stimulus presented at 1 location * some uncertainty associated with it, on average correct but uncertainty over their judgement * smaller blob - can be more certainty as more sure on accuracy * larger blob - high uncertainty

Question 70

costs of erroneous multisensory integration

Answer 70

when sensory cues relating to different sources, integrating them wouldnt be beneficial

Question 71

balancing the benefits and costs | multisensory integration

Answer 71

* integration is restricted to multisensory signals that occue close together in space and time * the degree of integration is shown by the amount of weight given to the visual stimulus * 0 indicates no integration * 1 signifies indicates complete visual capture ## Footnote * integrate things that aren't too different from each other * weight given to vision highest when discrepancies are quite small

Question 72

multisensory recalibration | what happens if there is a consistent discrepancy between sensory estima

Answer 72

* neurons in the optic tectum of the barn owl respond to both auditory and visual stimuli and have co-localised spatial receptive fields * if juvenile barn owls are made to wear prism goggles that shift the visual image laterally, auditory receptive fields shift so as to counteract the introduced audio-visual discrepancy ## Footnote * barn owl has good spatial hearing: has optic tectum * sound will arrive at ears at different times if not straight on * can use this to discover location * spatial receptive fields * adaptation??? Over 28 days so long time period, also in juvenile owls

Question 73

spatial recalibration | multisensory adaptation

Answer 73

* after exposure to a sequence of audio-visual stimuli with a consistent spatial offset, the perceive locations of auditory stimuli are shifted in the direction of the conflict * this effect can be elicited with as little as one presentation of the discrepant stimulus, suggesting the brain is constantly recalibrating auditory and visual position ## Footnote * simultaneous visual and auditory stimuli * location of sounds starts to get pulled to the right * almost shifted enough to cancel out the discrepancy * brain constantly monitoring the consistency/discrepancy of multiple modalities

Question 74

temporal recalibration | multisensory adaptation

Answer 74

* repeated exposure to pairs to auditory-visual stimuli with a consistent lag changes the perceived timing of subsequently presented stimuli * e.g. following adaptation sequences in which a visual flash precedes an auditory click by 100ms, a small visual lead might be required for perceived simultanityt

Question 75

perceptual latency account | mechanisms of temporal recalibration

Answer 75

* asynchrony adaptation changes the speed at which sounds are processed * reaction times to auditory stimuli speed up to counteract a visual lead and slow down to counteract an auditory lead * predicts a uniform recalibration of perceived timing

Question 76

roach et al. (2011) | mechanisms of temporal recalibration

Answer 76

* measured effects of asynchrony adaptation on perception of a wide range of stimulus onset asynchronies * rather than being uniform, changes in perceived timing vary systematically as a function of the difference between adapted and tested SOA

Question 77

population coding account | mechanisms of temporal recalibration

Answer 77

* results are consistent with a model in which audio-visual timing is represented by neurons tuned to different SOAs * this is the same type of explanation as used for tilt and direction aftereffects, suggesting that multisensory timing might now be processed in an analogous manner to simple unisensory properties

Question 78

neural mechanisms - superior colliculus

Answer 78

* multisensory integration has been most thoroughly studied in the SC * cells in the superficial layers are purely visual, but cells in the deep layers are often bimodal and sometimes trimodal * a hallmark of multisensory neurons in SC is super-additivity: the response to bimodal or trimodal input is often greater than the sum of the unimodal responses

Question 79

spatial rule & SC

Answer 79

receptive fields of different sensory inputs to a given multisensory neuron in SC are spatially aligned with one another, so maximum response is achieved only with co-located stimuli

Question 80

temporal rule & SC

Answer 80

inputs must also be approximately aligned in time

Question 81

inverse effectiveness rule & SC

Answer 81

super-additivity is strongest when responses to modality-specific input is weak

Question 82

neural mechanisms - association cortex

Answer 82

* a variety of 'association' areas have been identified in the cerebral cortex, which receive converging inputs from multisensory modalities * posterior portions of superior temporal sulcus * regions of posterior parietal cortex

Question 83

neural mechanisms - primary sensory cortex

Answer 83

* accumulating evidence is challenges the traditional modular view of primary sensory cortices as functionally independent and sensory specific * direct anatomical connections have been found between primary visual and auditory cortex * functional imaging results suggesting that auditory cortex is activated during silent lipreading * cross-modal recruitment of occipital visual cortex in the blind and auditory cortex in the deaf