ML

Question 1

What is Machine Learning?!

Answer 1

Machine learns from data …

Question 2

Machine Learning: Definition

Answer 2

• ML is about enabling a machine to act, without explicitly programming it!
• It is predicting unknown from uncertain information
• Self-configuring data structures that allow a computer to do things that would be called “intelligent” if a human did it ….
• “making computers behave like they do in the movies”

Question 3

Data is recorded from some real-world phenomenon. What might we want to do with that data?

Answer 3

Prediction - what can we predict about this phenomenon?

Description - how can we describe/understand this phenomenon in a new way?

Question 4

Learning from Data: What is it Good for?

Answer 4

Right: Write code to make the computer learn how to do the tasks
Wrong: Write code to explicitly do the above tasks

Question 5

Many applications are immensely hard to program directly. These almost always turn out to be “pattern recognition” tasks.

Answer 5

Right: 1. Program the computer to be able to learn from examples. 2. Provide “training” data.
Wrong: 1. Program the computer to do the pattern recognition task directly.

Question 6

Steps Involved in Software Engineering Methodology

Answer 6

• Analyze
– Interview experts, users, etc. to determine the actions the system must perform
• Design
– Apply Computer Science knowledge to design a solution
• Implement
• Test

Question 7

Challenges for Software Engineering Methodology

Answer 7

• Standard SE methods fail when…
– System requirements are hard to collect
– The system must resolve difficult tradeoffs

Question 8

Why System Requirements Can be Hard to Collect …

Answer 8

• There are no human experts
– Cellular telephone fraud
• Human experts are inarticulate
– Handwriting recognition
• The requirements are changing rapidly
– Computer intrusion detection
• Each user has different requirements
– E-mail filtering

Question 9

Machine Learning:

Answer 9

Replacing Guesswork with Data
• In all of these cases, the standard SE methodology requires engineers to make guesses
• Machine Learning provides a way of making these decisions based on data, instead of guesswork

Question 10

Machine Learning vs. Artificial Intelligence

Answer 10

Humans can think, learn, see, understand language, reason, etc.
• Artificial Intelligence aims to reproduce these capabilities.
• Machine Learning is one part of Artificial Intelligence.
• Machine learning aims to reproduce learning capabilities

Question 11

Using machine learning to detect spam emails.

Answer 11

Naïve Bayes

Rule mining

Question 12

Using machine learning to recommend books.

Answer 12

Collaborative Filtering
Nearest Neighbour
Clustering

Question 13

Using machine learning to identify faces and expressions.

Answer 13

Decision Trees

Adaboost

Question 14

Using machine learning to identify vocal patterns

Answer 14

Feature Extraction
Probabilistic Classifiers
Support Vector Machines

Question 15

ML for working with social network data:

Answer 15

detecting fraud, predicting click-thru patterns, targeted advertising, etc etc etc .

ALGORITHMS
Support Vector Machines
Collaborative filtering
Rule mining algorithms

Question 16

Usually First Step in Machine Learning

Answer 16

Feature Extraction

Question 17

Feature Extraction

Answer 17

presenting stimulies using their distinctive attributes

Question 18

Feature Extraction: What for?

Answer 18

Decreasing the size of data presented to machine
– Saving computational and memory load
Presenting stimuli’s using their discriminative attributes
– Not including attributes which does not contribute in the specific learning task.
– Not including attributes which does not help in distinguishing between different stimulies.
– Example: Transcribing speech to text:
• Loudness is irrelevant to message content, no matter how load you speak, you still say the same thing!
• Loudness information should be removed in the feature extraction process
– Removing attributes helps in getting rid of extra sources of noise
• Example: when loudness is removed, no need to deal with noise coupled with loudness …

Question 19

Features should be…

Answer 19

discriminative between different stimuli classes: e.g. discriminating between apple and orange!

Question 20

Two steps of ML:

Answer 20

– Training: Letting the machine learn association between a stimuli class and its features
– Testing: Presenting a new stimuli to machine, asking the machine to recognize the stimuli class!

Question 21

Forms of Learning:

Answer 21

Supervised and Unsupervised Learning

Question 22

Supervised Learning

Answer 22

– Agent learns a function mapping input to output from examples, which are labeled into known classes

Question 23

Unsupervised Learning

Answer 23

– Agent identifies patterns in examples, but no known label is provided

Question 24

Perceptrons (Neural Networks)

Answer 24

• Perceptron model simulates brain cognitive mechanism
• Perceptron is a dense interconnection of computing cells, similar to brain which is a dense interconnection of biological cells

Question 25

Perceptron unit

Answer 25

A Perceptron is a dense interconnection of below units.
Input links → Input function → activation function→ output → output links

Inputs to the cells are weighted individually (multiplied by weights), summed up, and then pass through a nonlinearity in order to form the output

Question 26

Perceptron is a Feed Forward Network

Answer 26

Outputs of each layer is calculated based on outputs of previous layers

Question 27

Propagation

Answer 27

Computations in each layer is propagated to the next layer:
• In the first layer, features are multiplied by weights, summed and passed by a non-linear function in order to form outputs of f3 and f4 …

Question 28

Perceptron Learning

Answer 28

• Usually in machine learning, an error (cost) function (E) is defined to be minimized.
– The error function is distance between the output of the machine and the known labels.
– Example: A robot has to reach a certain position, the error function is the distance between the target position and its final position.
• Learning involves adjusting weights in the network (parameters in the model), so that the error function is minimized!

Question 29

Gradient descent is a …

Answer 29

first-order iterative optimization algorithm for finding the minimum of a function. To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient (or of the approximate gradient) of the function at the current point.
(Gradientenverfahren, auch Verfahren des steilsten Abstiegs)