week 5 - efficient coding III Flashcards

(34 cards)

1
Q

What is the ECH? (two parts)

A

a group of neurons should encode as much information as possible OR remove as much redundancy as possible

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What is the equation for ECH? Maximize…?
What does each component mean?

A

Maximize I(S;f(S))
I = mutual information
S = signal
f(S) = tuning curves to be optimized

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What are the efficient coding (EC) parts of Whitening and ICA?

A

Whitening
EC = decorrelating pixels/data
ICA
EC = demixing to recover independent non-Gaussian components

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

According to ECH, why do we need to reverse the mixing process (ICA)?
Explain this with visual sensory neurons, V1, input, output?

A

EC: RESPONSES of neurons should be independent

input to sensory neurons is correlated but sensory neurons decorrelate it (whitneing). then brain mixes up inputs. inputs are then demixed to make motor neurons responses independent (ICA)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What model does ICA build apon?

A

ICA builds on (localized) whitening model

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is the process of ICA?

A

-demix signals by finding the directionfo the data whichis the least Gaussian -> to recover independent non-Gaussian components -> neural responses are independent (aligns with EC)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Sparse Coding:
What is the problem with local codes? (assign one neuron to a concept)
What is the problem with dense codes? (assign one concept to many neurons)
What is the solution to these two problems ^?

A

-when this neuron dies, do you forget about this concept? no eg. grandma
-very robust however would cost a lot of energy (all activate at same time)

-use spare, distributed codes
sparse activation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What are the benefits of sparse, distributed codes?

A

maximalise memory storage but also save energy

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What is kurtosis?
What does kurtosis describe?

A

a statistical term which describes the shape of the probability distribution curve
it describes the ‘taildness’, the prescence of outliers and shape of the peak

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What do probability distributions with positive kurtosis look like?

A

sharp peak
heavier tails/more outliers

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What is positive kurtosis aka?

A

super-Gaussian
leptokurtotic

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What is the equation for the encoding model?
What does each component mean?

A

r = Ws
r=neural responses
W=receptive field weights
s=natural image pixels (signal/image data)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What is the decoding model equation?

A

s=W(-1)r

(-1) is to the power of minus 1

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What is the equation for the sparse coding model?
What does each term mean?

A

minimise E = -[preserve information] - λ[spareness]

E= energy used

preserve information = the error term (men squared error)

spareness = a function which penalizes non-zero values

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What is the goal of sparse (distributed) coding?

A

maximize memory storage
save energy (minimise energy used?)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

In the sparse coding model equation, what does is the preserve information term mean mathematically?

A

preserve information = mean squared error

(this is the reconstruction error)

17
Q

In the sparse coding model equation, what type of function represents the sparseness term?

A

sparseness = a function that penalizes NON-zero values

18
Q

What do sparse filters look like?
What characteristics do they have?

A

like receptive fields in primary visual cortex V1
they are localised, orientation-specific and Gabor-like

19
Q

What other type of filters do sparse filters look like?

A

like ICA filters

20
Q

Which two filters look like V1 receptive fields?

A

sparse filters
ICA filters

21
Q

What is the difference between the data/images used in ICA and in sparse coding?

A

ICA = mix of independent components with non-Gaussian stats
Sparse = has super-Gaussian response statistics

22
Q

What does the sparse filter do?

A

maximalise sparseness

23
Q

Will the neural response properties of ICA be Gaussian?

A

NO! non-Gaussian (recover independent non-Gaussian components)
as it is the task of the brain/neural response to demix the signal

24
Q

Why is it desirable to have super-Gaussian statistics for the neural response in sparse coding?

A

because super-Gaussian stats are desirable because they maximalise information even when there are energy constraints in the nervous system

25
What is a criticism of the definition for sparse coding?
it is a bit vague: Are neurons’ responses SPARSE across population or time?
26
To criticise sparse coding, how is it overly simple?
the brain is more complicated than binary networks eg. brain has inhibitory and excitatory neurons
27
What is a criticism of the main focus of sparse coding?
it focusses on memory storage as the limiting factor of the brain but maybe generalization is more important to focus on
28
How to neural responses react to natural sounds (birds, waves) and speech in humans?
neural responses are specifically adapted/tuned to natural sounds, but very similar tuning properties emerge for speech sounds!
29
Potentially, what does the strength of cortical magnification depend on? What area have high cortical magnification?
strength of cortical magnification depends on neural resource limits Strong magnification: particular sensory region receives an exceptionally large number of neurons in cortex RELATIVE to its physical size or sensory receptor count -fovea, fingertips
30
How can efficient coding be applied to dynamic problems in the human body?
problem: olfaction receptors can regenerate efficient coding can be applied to this dynamic problem
31
How does the ECH run into problems when applying it to behaviour (in humans)?
EC maximalises ALL information indiscriminately however this is not true as some stimuli are more behaviourally relevant than others. For example, a human would react more to a tiger than a flower. Thus ECH is not representative of natural human behaviour
32
What is reverse efficient coding?
-calculate (presumed) stimulus statistics from neural responses -this is instead of calculating the optimal neural responses from stimulus statistics
33
What the are the assumptions needed for reverse efficient coding?
you can do reverse EC assuming that these stimuli are encoded efficiently
34
What does the curve look like for reverse coding when stimulus is on the xaxis and neural response on the yaxis? What does it show?
sigmoidal - as stimuli increase then