Week 3 Flashcards
what is meant by hierachical representation in ANN’s?
hierarchical representation= the way these networks process and represent data through multiple layers, where each layer captures different levels of abstraction. This concept is a key factor in the success of deep learning models, particularly for complex tasks like image and speech recognition.
What are the two main types of problems addressed by supervised learning?
-
classification, which involves assigning discrete labels to inputs, -
regression, which involves assigning a continuous, real-valued output to an input.
A form of supervised learning is a decision tree: What are examples of features used in decision trees for classification?
Features can include categorical data like restaurant type (French, Thai, Italian), price category ($, $$, $$$, $$$$), occupancy (quiet, busy), date quality (not so interested, very interested), and binary questions like “Hungry?” or “Raining?”.
What principle guides the selection of questions in a decision tree?
The questions chosen should maximize information gain.
ANNs are composed of a network of interconnected units or neurons, organized in layers: input, hidden, and output.
How can an ANN function as a classifier?
With a certain set of connection weights, the network can classify data, like determining if a group of inputs represents siblings or acquaintances.
How are the weights in an ANN determined?
The weights are determined through supervised learning, where the network is trained with data that has known outputs.
What method is used for supervised learning of weights in ANNs?
Backpropagation of error is used, where weights are assigned randomly and adjusted stepswise based on the error in output.
What are the 3 steps involved in backpropagation?
- Initialize with random weights.
- For each instance, compare network output to required output.
- Adjust weights to reduce error, working back from the output layer to input.
When are ANNs particularly successful?
ANNs excel when given large amounts of data, such as outperforming experts in diagnosing skin lesions.
deep networks in ANN are?
networks with more layers than just input and output.
Does adding more layers to an ANN always improve performance?
No, the improvement depends on the task. Deep networks enable hierarchical representation, which, along with techniques like convolution, contributes to their success.
what is convolution?
convolution= mathematical operation that combines two functions to produce a third function. It expresses how the shape of one is modified by the other.
what are the two main risks accompanied by simply adding more neurons to an ANN?
-
overfitting=Overfitting occurs when the model becomes so complex that it
starts to learn noise and random fluctuations in the training data as if they were meaningful patterns,leading to poor performance on new, unseen data. - diminishing return=after a certain threshold, the benefits can taper off, while the costs (in terms of overfitting risk, computation, and training data requirements) continue to rise.
What is required beyond adding more neurons to improve an ANN’s performance?
Specific changes in the network’s topology, such as creating deep networks, are needed to enhance its function and performance.
if we use instance-based learning and we ask this question and there are no x’s close. What do we do?
k-nearest neighbors (KNN):
How does the k-NN algorithm determine the neighbors of a query point?
How does k-NN perform classification?
How does k-NN perform regression?
What are the four steps in k-mean clustering?
