Perception 1: Low level vision

Question 1

What is the receptive field of a neuron?

Answer 1

Part in visual space where stimulus give rise to stimulating a neuron - receptive field of a neuron
Excite/inhibit ganglion cell

Question 2

How are ocular dominance columns (right or left eye dominant) arranged in relation to orientation columns? (columnar arrangement in V1)

Answer 2

Perpendicular

Question 3

What is a hypercolumn? (in V1)

Answer 3

A cortical processing module for a stimulus that falls within a particular retinal area

Formed from one set of ocular dominance columns and one set of orientation columns

Question 4

What is feature detection theory of visual processing?

Answer 4

As you go through ventral stream, cells respond to larger and more complex stimuli
Hierarchy of cells

e.g.
V1 - edges and lines
V2 - shapes
V4 - objects
IT - faces

(receptive field size increases)

Question 5

What two variables can affect spikes per second? (response rate in a particular cell)

Answer 5

Orientation and contrast

Question 6

What is the Fourier analysis framework?

Answer 6

Visual system deconstructs an image into discrete channels
Each channel conveys information contained in the image at a specific spatial scale and orientation

High spatial frequencies (low scale) for visual detail
Low spatial frequencies (high scale) for broad structure

Visual system then recombines them to form a coherent representation of the scene

Question 7

What is coarse vs fine luminance?

Answer 7

Coarse luminance changes in an image - reveal large-scale structure

Fine luminance changes in an image - reveal small-scale detail

Question 8

What happens if you remove high vs low spatial frequency content from an image?

Answer 8

High removed = coarse (blurry)
Low removed = fine (detailed but no sense of bigger picture)

Info present at different scales within the same image

Question 9

What does the mathematical theorem behind Fourier analysis state?

Answer 9

Any complex signal can be constructed from a set of simpler sinusoidal functions

So, vision can be broken down into simpler parts (frequency and contrast) to explain its complexity

Question 10

What does contrast sensitivity vary as a function of?

Answer 10

Spatial frequency

This is the contrast sensitivity function (CSF)

Question 11

What is the contrast sensitivity function?

Answer 11

How well you can see detail across a full range of spatial frequencies

Question 12

When is sensitivity (CSF) best and why?

Answer 12

Sensitivity is maximum between 2-5 cpd

Sensitivity is better at central range of spatial frequencies as you need less contrast in the image to resolve a pattern
At extremes of spatial frequency - more contrast is needed to resolve a pattern

More sensitive = you don’t need as much contrast to perceive something

Question 13

What did Blakemore and Campbell want to show with their spatial frequency adaptation experiment?

Answer 13

Show that someone’s sensitivity can change within a narrow range of spatial frequency as a result of adaptation

Question 14

What is contrast threshold?

Answer 14

The contrast of an image, below which the pattern looks homogenous (cannot see any detail)

Question 15

What was Blakemore and Campbell’s (1969) spatial frequency adaptation experiment?

Answer 15

Participants set the contrast of an image (grating - stripes) so that they could just perceive luminance differences

Then, they introduced a high contrast adapting stimulus (clear stripes) for 60 secs
- This was at a specific spatial frequency
- Neurons responding to this dull and habituate, becoming less sensitive

Then, participants redo the first task (for contrasts at the same spatial frequency)

Question 16

What happens to contrast threshold after adaptation? What does this show about neurons involved? (Blakemore and Campbell)

Answer 16

It increases, contrast has to be higher for people to perceive differences
Harder to see stimuli at a lower threshold

Higher threshold = easier to see stimuli

Shows that neurons for that spatial frequency habituated to adaptive stimulus and became less sensitive

Question 17

After adaptation, how do threshold and sensitivity change in response to spatial frequencies similar to the adapting frequency?
What does this selective adaptation effect imply the existence of?

Answer 17

Threshold is increased
Sensitivity is decreased

This selective adaptation effect implies the existence of multiple, overlapping, spatial frequency channels

Question 18

What electrophysiological evidence supports conclusions from the spatial frequency adaptation effect - that there are multiple, overlapping, spatial frequency channels?

Answer 18

Contrast sensitivity functions of V1 cells in macaque monkey
Acquired by drifting sinusoidal gratings over receptive fields and measuring response
All acquired from the same location on retina

Cells respond preferential to specific spatial frequency bands - different but overlapping spatial frequency selectivities, they do Fourier analysis of the visual image

Question 19

What are David Marr’s 3 levels of analysis that must be applied in order to understand how a system works?

Answer 19

1) Computational level
What is the goal of the system?
i.e. what is purpose of vision - could be to recognise an object

2) Algorithmic level
What rules and representations can achieve this goal?

3) Implementational level
How is it achieved physically?

Vision serves multiple goals, including determining where objects are, what shape they have, and how to interact with them.

Each of these goals relies on numerous algorithmic steps.

Question 20

Edge detection is one of the algorithmic steps that helps us to determine what objects are and what shape they have.
Where do edges exist?

Answer 20

They only exist in interaction between observer and image
We focus on edges even when they are not inherently contained in an image

Question 21

What are the four steps in Marr’s model of object perception?

Answer 21

Gray-level - photoreceptors
Primal sketch - identifies object boundaries (edges)
2.5D sketch - depth perception
3D model

Question 22

What does Marr and Hildreth’s model of edge detection assume?

Answer 22

Assumes that edges of an object coincide with gradients in luminance
Only works if coincided luminance change where there is an edge
e.g. luminance gradient would seperate lake from tree

Question 23

What is involved in Marr and Hildreth’s (1980) model of edge detection - what are the steps? (first and second derivative)

Answer 23

Luminance gradient between two things
e.g. dark tree on light lake background

Take the first derivative of this - shows us sharp luminance changes that correspond to peaks/valleys where there is an intensity gradient
First derivative = rate of change in signal (across edge)

The second derivative is then taken, to stop signal being susceptible to noise - otherwise could be difficult to decide where peaks and valleys are

The second derivative gives us zero crossings (luminance changes across an edge - goes from negative to positive) where there is a luminance gradient
Edge not represented by peak or valley anymore

Question 24

What does Marr and Hildreth’s model allow computers to do?

Answer 24

Created algorithm so that computers can change image so only edges are highlighted

Question 25

Why is there a smoothing process in edge detection?

Answer 25

This process will be negatively affected by high frequency noise - zero crossings where no meaningful gradient exists - detect edges where they are not actually present

Question 26

What happens in the smoothing process of edge detection?

Answer 26

- Equivalent to first blurring the image (pre-derivatives) - Equivalent to removing high frequency content (i.e. analysis at coarse spatial scale) Expressed as convolving the image with a Gaussian operator G - each pixel is blurred with its neighbours - the sigma of G determines the level of blurring Higher level of sigma = more blurry image More blurry = more accurate edge detection, as edge detection is not corrupted by random noise

Question 27

Does edge detection happen sequentially? i.e. Luminance change Smoothed First derivative Second derivative - zero crossing shows where edge is

Answer 27

No This can be done in all one step: Simple filtering operation - convolving the original image with a Laplacian of Gaussian filter achieves the same steps in one operation

Question 28

How is the Laplacian of Gaussian filter implemented at a biological level?

Answer 28

2D model of it - rotationally invariant When this is plotted top-down It represents a retinal or LGN receptive field!

Question 29

The smaller the LoG filter, the smaller the...

Answer 29

Spatial scale

Question 30

What should LoG filters span?

Answer 30

A range of sizes so that the full range of scales and spatial frequencies are sampled There is a trade-off between noise removal (better at coarser scales) and edge enhancement (better at finer scales)

Question 31

What is the spatial coincidence rule?

Answer 31

“If a zero-crossing segment is present in a set of independent channels (scales) over a continuous range of sizes… then the set of such zero crossing segments may be taken to indicate the presence of an intensity change in the image that is due to single physical phenomenon (a change in reflectance, illumination, depth or surface orientation)”. Marr and Hildreth

Question 32

How do coarse and fine scales tell us if there is a meaningful edge?

Answer 32

Coarse scale = where edges are Finer scale = if edges in same place, edge information is meaningful Can use these to more accurately detect position of image Combining info from different spatial scales to get the trade off and best of both Retinal and LGN receptive fields are spatial filters that compute the second derivative of an image

Question 33

How do off and on centre cells both inform simple cells in V1 of where an edge is?

Answer 33

Location of zero crossing is represented in both on and off centre cells Both cells excite simple cell - simple cell responds when both cells give a positive response Combine info to localise presence of edge - zero crossing

Question 34

How does rapid edge detection (Paradiso and Nakayama (1991) show that low level visual perception is mostly edge based?

Answer 34

A target (white circle with black background) is presented for 16ms A mask (white ring on black background (smaller than white circle) - inner bit of circle is black) is then presented for 16ms Observer sees composite image of both target and mask Observers rate the brightness of the outside high and brightness of the inside dark Shows that observers see edges first, then rest of image is filled in - filling in process was interrupted by mask This shows that low level visual perception is mostly edge based

Question 35

What are first order vs second order edges?

Answer 35

First order - defined by luminance gradient Second order - no luminance difference, defined by texture

Question 36

What was Julesz's texton account of texture edge perception?

Answer 36

Julesz (1981) developed a model based on statistics of local conspicuous image features - textons Based on the simple idea that differences in the statistics of certain kinds of “conspicuous” features means certain features jump out at you - Oriented lines - Line terminations - Junctions (T and X junctions) Based on feature detection theory and preattentive visual search

Question 37

What is the difference between effortless and difficult segmentation (Bergen and Julesz, 1983)

Answer 37

Effortless - difference in conspicuous element (e.g. line crossings) Difficult - no difference in conspicuous elements

Question 38

How did Nothdurft (1985) critique texton account of texture edge perception?

Answer 38

Noticed similarities between luminance segmentation and texture segmentation Luminance segmentation becomes more difficult as element spacing increases - when textons are more spaced out, it is harder to detect a shape among the textons Even when features of textons are kept the same, segmentation ability can vary considerably Instead, these results suggest a mechanism that is more similar to an edge detection mechanism sensitive to spatial scale (i.e. spatial frequency) and orientation

Question 39

What is Bergen and Adleson's (1988) Texture gradient theory of texture segmentation?

Answer 39

Evaluation of gradient (as for luminance) allows for texture segmentation Closer to Marr and Hildreth theory Making some textons longer = easier Making some textons shorter = harder - This is because it involves changing spatial frequency content of the image Different spatial scales = how we can do texture segmentation Computational models of texture segmentation (or edge detection) now explain this not in terms of textons but in terms of spatial differences in orientation and spatial frequency statistics This brings us back to “Fourier analysis”, in which an image is represented in terms of energy contained within channels tuned to combinations of spatial frequency and orientation

Question 40

How does texture segmentation by texture gradient theory map onto neurophysiology? Lamme et al (1999)

Answer 40

Lamme et al (1999) used single cell recording to record from neurons in V1 in awake macaque monkey Receptive field was mapped and activity was recorded at different positions along the texture Temporal sequence of neural responses revealed separate processes (right figure shows fig – ground response): - Initial response (50 ms) - Local orientation tuning (58 ms) - Enhancement at edge (90 ms) - Enhancement in figure (112 - 123 ms) Consistent with an edge-based segmentation process that drives filling in of texture surface Edge enhancement at 90 ms suggests feedback from beyond V1. Only immediate sensitivity to the edge, with feedback from other visual areas Thus, low level vision is edge detection