Connectionism Flashcards
(49 cards)
Neural network
Consists of an input layer, hidden layers and an output layer
Overall the network transforms any input pattern into a corresponding output pattern as dictated by the arrangement and strength of the many connections between neuro
Chinese gym
Many Chinese room’s might solve the symbol grounding problem (but we need to many since 1 neuron = one person)
Gradient descend
An optimization algorithm for finding a local minimum of a differentiable function
Used for back-propagation
Solution to symbol grounding problem
Rises from complexity.
How we assign meaning lays beyond language and humans
One must look at how neurons code and transform sensory signals
the relations between weights and activations of the neurons of the trained model
Weights
How much one neuron is firing to another (the lines that connect the units)
Activation
If a neuron/unit is activated (or not)
Analogue
On/off
Parallel processing
Instead of having serial processing (one at a time) one has multiple at a time
Back-propagation
Learning neural networks by gradient estimation (optimization algorithm)
The strategy exploits the calculated error between the actual values of the processing units in the output layer and the desired values, which is provided by a training signal. The resulting error signal is propagated from the output layer backward to the input layer and used to adjust each weight in the network. The network learns, as the weights are changed, to minimize the mean squared error over the training set of words
Fundamental research program of classical AI
Identify the undoubtedly complex function that governs the human pattern of response to the environment and then write the program (the set of recursively applicable rules) by which the SM machine will compute it.
Classical AI
It’s like a programmed robot following a strict set of human-made rules
Anatomic points in the brain that inspired connectionism
Nervous system: Parallel machines
The neurons are comparatively simple: Analog response
- They are somewhat digital - it is firing or not
- We look at the firing frequency and translate it into a value between 0 and 1
- This is a contrast to computers 0’s and 1’s
In the brain, axons projecting from one neuronal population to another are often matched by axons returning from their target population
Function (connectionism)
Any vector-to-vector transformation
Materialism
Everything is physical
Dualism
Distinction between body and mind
NETtalk
It converts English text to speech sounds
1. Receives as input letter in a word
○ Local representation
2. Performs the transformation
3. Yield the elementary speech sounds
Has 309 processing units and 18,629 connection strengths (weights) that must be specified. The network does not have any initial or built-in organization for processing the input or (more exactly) mapping letters onto sounds. All the structure emerges during the training period. The values of the weights are determined by using the “back-propagation” learning algorithm developed by Rumelhart, Hinton, and Williams (1986)
SM machine
Symbol manipulating machine
Church’s thesis
Every effective computable functions is recursive computable
Effectively computable
There is a “rote” procedure for determining, in finite time, the output of the function for a given input
Recursively computable
There is a finite set of operations that can be applied to a given input, and then applied again and again to the successive results of such applications, to yield the function’s output in finite time
The Luminous room (Maxwell)
Because something is valid it doesn’t need to be sound
Maxwell and light (theoretical)
Axiom 1. Electricity and magnetism are forces.
Axiom 2. The essential property of light is luminance.
Axiom 3. Forces by themselves are neither constitutive of nor sufficient for luminance.
Conclusion 1. Electricity and magnetism are neither constitutive of nor sufficient for light.
Problem: If you stand in a dark room with a magnet there will be no light
Maxwell’s responses:
The “luminous room” experiment is a misleading display of the phenomenon of luminance because the frequency of oscillation of the magnet is absurdly low, too low by a factor of 10^15.
But the room already contains everything essential to light according to Maxwell’s own theory
Main problems with the Chinese rom
What people can or cannot imagine often has nothing to do with what is or is not the case, even where the people involved are highly intelligent
Axiom 3 is false
The failure of classical AI
The functional architecture of classical SM machines is simply the wrong architecture for the very demanding jobs require
Process training up network
There are various procedures for adjusting the weights so as to yield a network that computes almost any function-that is, any vector-to-vector transformation-that one might desire. In fact, one can even impose on it a function one is unable to specify, so long as one can supply a set of examples of the desired input-output pairs.
