Brainscape's Knowledge GenomeTM


Top Flashcards (Certified)
All Flashcards

AI > Lecture 4 Lecture Notes > Flashcards

Lecture 4 Lecture Notes Flashcards

(74 cards)

Start Studying
1
Q

How do McCulloch and Pitts neurons function?

A

They sum the firing of incoming neurons multiplied by synapse weights and fire if the sum exceeds a threshold.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What does a perceptron consist of?

A

Sensory neurons connected to motor neurons.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What is the main learning rule for perceptrons?

A

Adjust weights based on the difference between actual and desired outputs.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What is the significance of the bias unit in a perceptron?

A

It allows the perceptron to create any dividing line needed for classification.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What boolean functions can perceptrons learn?

A
  • AND
  • OR
  • NAND
  • NOR
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is a limitation of perceptrons?

A

They can only learn linearly separable boolean functions.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Which boolean function cannot be learned by a simple perceptron?

A

XOR.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What is the equation used in perceptrons for output calculation?

A

~oi = step(ÂWij~xj).

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What is a common value for the learning rate in perceptrons?

A

0.1.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What happens if the output neuron incorrectly produces a 1?

A

Decrease the weight for that neuron.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What happens if the output neuron incorrectly produces a 0?

A

Increase the weight for that neuron.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What are the outputs of a perceptron when given the inputs (0,1) and (1,0)?

A

1’s

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What are the outputs of a perceptron when given the inputs (0,0) and (1,1)?

A

0’s

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What type of functions can perceptrons learn?

A

Boolean functions which are linearly separable

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

For a 2-variable input, how can the 1’s and 0’s be divided?

A

With a straight line

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

For a 3-variable input, how can the 1’s and 0’s be divided?

A

With a plane

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

What did Minsky and Papert predict about advanced forms of perceptrons?

A

They were unlikely to escape the problem of linear separability

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What was the impact of Minsky’s reputation on the field of neural networks?

A

It wiped out the entire field for over a decade

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Can multilayer neural networks learn functions beyond linear separable ones?

A

Yes, they can learn any function

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What is the perceptron considered in terms of classification?

A

A binary classification algorithm

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

What does a neural network do beyond classification?

A

Learns a continuous function from one multidimensional space to another

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

What are the two generalizations made to the single-layer perceptron?

A
  • Change the step function to a differentiable function
  • Define a formal learning algorithm in terms of gradient descent
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

Why must the initial weights of a multilayer neural network be small random values?

A

If all weights are 0, the network cannot learn

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

What is the delta rule used for in neural networks?

A

Modifying weights based on their contribution to the final outcome

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
What is the sigmoid function defined as?
s(u) = 1 / (1 + e^(-bu))
26
What does the parameter 'b' affect in the sigmoid function?
The slope of the curve
27
What is the derivative of the sigmoid function?
s'(u) = s(u)(1 - s(u))
28
What do multilayer neural networks allow for in terms of function learning?
Learning any continuous, differentiable function
29
What is the role of the hidden layer in a multilayer neural network?
It allows for more complex function learning
30
How is the output of a neuron in a multilayer network defined?
o_i = s(Σ(W_ij h_j))
31
What is the purpose of the backpropagation learning rule?
To adjust weights based on the error in output
32
What does the error metric E represent?
E = 1/2 Σ(y_i - o_i)²
33
What happens if you increase the number of neurons in the hidden layer too much?
The network may overfit and not generalize well
34
What is the goal of training a neural network?
To minimize the error metric
35
True or False: Backpropagation guarantees convergence for all input cases.
False
36
What effect does lowering the learning rate have on a neural network?
It takes longer to learn but increases the chances of finding the global optimum
37
What is the overall procedure for training a backpropagation neural network?
* Pick an input/expected output vector pair * Present the input to the network * Read the network’s output * Modify each weight using the delta learning rule * Repeat
38
What is the significance of generalization in neural networks?
It allows the network to apply learned rules to unseen inputs
39
What is the relationship between the delta rule and gradient descent?
The delta rule moves weights in the direction that reduces error based on the gradient
40
What is the error metric required to be zero for zero error?
It must have a mean squared error form
41
What happens when the network is caught in a suboptimum solution?
It may fail to converge for some inputs
42
What is the backpropagation learning rule used for?
To update weights in a neural network based on the error between predicted and actual outputs ## Footnote The rule helps in minimizing the error during training by adjusting weights accordingly.
43
What is the formula for updating weights DWij?
DWij = a(~yi - ~oi)~oi(1 - ~oi)hj ## Footnote Where a is the learning rate, ~yi is the target output, ~oi is the actual output, and hj is the hidden unit.
44
What does DVjk depend on in the backpropagation algorithm?
The value of Wij ## Footnote This means that Wij should not be changed until after DVjk is computed.
45
How is the weight update DWij expressed in matrix form?
DW = a ((~y - ~o) ⌦ ~o ⌦ (1 - ~o)) h ## Footnote This represents the element-wise multiplication in the weight update process.
46
What is the purpose of the stopping criterion 'd' in the error backpropagation algorithm?
To stop the algorithm when the outputs of the neural network aren't changing significantly anymore ## Footnote This criterion helps avoid unnecessary iterations once convergence is achieved.
47
What is the learning rate 'a' used for in weight updates?
It controls how much the weights are adjusted during each update ## Footnote Smaller values are typically preferred to ensure stable convergence.
48
What is a Hopfield network used for?
To simulate associative memory ## Footnote It allows the retrieval of stored patterns based on partial or noisy inputs.
49
What does Hebb's Rule state about synaptic strength?
If neuron A fires and neuron B fires in response, the strength of the synapse between them increases ## Footnote This rule is foundational to understanding learning mechanisms in neural networks.
50
What is the difference between Hebb's Rule and the Delta rule?
Hebb's Rule updates weights based on correlation between nodes, while the Delta rule updates based on the error of the output node ## Footnote This reflects different learning strategies in neural networks.
51
What happens if weights in a Hopfield network are set to learn multiple patterns?
The weights are set to the sum of correlations for all items to be stored ## Footnote This allows the network to retrieve the closest memorized vector when presented with an input.
52
What is the capacity of a Hopfield network with N neurons?
Approximately 0.138N vector patterns ## Footnote Exceeding this limit leads to degradation in performance.
53
What is the first step in the error backpropagation algorithm?
Initialize the matrices V and W with small random values centered at 0 ## Footnote Proper initialization is critical for effective training.
54
Fill in the blank: The output of a two-layer neural network is computed using the formula _______.
s(W~h) ## Footnote Where ~h is the output from the hidden layer and W is the weight matrix.
55
True or False: The Hopfield network is a fully-connected, feedforward neural network.
False ## Footnote Hopfield networks are recurrent, meaning they allow connections between neurons that can loop back on themselves.
56
What is the learning rule for weights in a Hopfield network?
Wij = (1/N) Σ x(p)(i) x(p)(j) ## Footnote This rule sums the correlations of all input patterns to determine weight strength.
57
What is the capacity of a Hopfield network with N neurons?
About 0.138N vector patterns ## Footnote Exceeding this capacity leads to degradation in performance.
58
What happens when a Hopfield network stores more than its capacity?
Degradation occurs, causing similar learned patterns to mix together ## Footnote This is analogous to how human memory degrades under information overload.
59
What type of algorithm does a Hopfield network use?
Nearest-neighbor algorithm
60
What is competitive learning in neural networks?
Neurons compete to categorize inputs based on proximity ## Footnote The closest neurons to the input become the strongest.
61
What is the structure of a simple competitive learning network?
Input neurons feed into output neurons, which are self-connected and have inhibitory connections to each other
62
What does the output of the neurons in a competitive network depend on?
The sum of their inputs weighted by edge weights
63
What occurs when one output neuron dominates in a competitive network?
It is designated as the winner, while the outputs of other neurons decrease
64
What is the learning rule for the competitive network?
Weights are adjusted only for the winning output neuron based on the input
65
Fill in the blank: A neuron that never gains enough strength to win is known as a _______.
Dead unit
66
What is leaky learning in neural networks?
Updating weights of both winner and loser neurons, albeit to a lesser degree for losers
67
What is lateral inhibition?
Output neurons inhibit each other to compete for dominance ## Footnote This concept is derived from neuroscience and helps in pattern recognition.
68
What is feature mapping in competitive networks?
A type of lateral inhibition that only applies to nearest neighbors of the output neuron
69
What does a self-organizing map do in neural networks?
Clusters inputs into spatially related categories
70
What is Kohonen’s algorithm?
A neural network that clusters input data into output without lateral inhibition
71
What is the initial weight setting in Kohonen's network?
Small random values
72
How does Kohonen's network determine the winning neuron?
By selecting the neuron with the largest output value
73
What does the neighborhood function L(i, i*) in Kohonen’s algorithm represent?
The proximity of neuron i to the winning neuron i* ## Footnote It is similar to a bell curve centered at i*.
74
What happens to the weights of output neurons in Kohonen’s network based on their proximity to the winner?
Weights are modified more for neurons close to the winner and less for those further away

AI (6 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2025 Bold Learning Solutions. Terms and Conditions