ML with AWS

Question 1

Three main components of neural networks

Answer 1

1. Input Layer: This layer receives data during training and when inference is performed after the model has been trained.
2. Hidden Layer: This layer finds important features in the input data that have predictive power based on the labels provided during training.
3. Output Layer: This layer generates the output or prediction of your model

Question 2

Explain convolutional neural networks (CNNs) in modern computer vision

Answer 2

In these neural networks, the hidden layers are used to extract different information about images. We call this process feature extraction.

These models can be trained much faster on millions of images and generate a better prediction than earlier models.

Question 3

feature extraction

Answer 3

hidden layers are used to extract different information about images

Question 4

Image classification

Answer 4

is the most common application of computer vision in use today. Image classification can be used to answer questions like What’s in this image? This type of task has applications in text detection or optical character recognition (OCR) and content moderation.

Question 5

Object detection

Answer 5

is closely related to image classification, but it allows users to gather more granular detail about an image. For example, rather than just knowing whether an object is present in an image, a user might want to know if there are multiple instances of the same object present in an image, or if objects from different classes appear in the same image.

Question 6

Semantic segmentation

Answer 6

is another common application of computer vision that takes a pixel-by-pixel approach. Instead of just identifying whether an object is present or not, it tries to identify down the pixel level which part of the image is part of the object.

Question 7

Activity recognition

Answer 7

is an application of computer vision that is based around videos rather than just images. Video has the added dimension of time and, therefore, models are able to detect changes that occur over time

Question 8

Input Layer

Answer 8

The first layer in a neural network. This layer receives all data that passes through the neural network.

Question 9

Hidden Layer

Answer 9

A layer that occurs between the output and input layers. Hidden layers are tailored to a specific task

Question 10

Output Layer

Answer 10

The last layer in a neural network. This layer is where the predictions are generated based on the information captured in the hidden layers.

Question 11

Machine Learning workflow review

Answer 11

1. Define the problem
2. Build the dataset
3. Train the model
4. Evaluate the model
5. Use the model

Question 12

reinforcement learning (RL)

Answer 12

an agent is trained to achieve a goal based on the feedback it receives as it interacts with an environment. It collects a number as a reward for each action it takes. Actions that help the agent achieve its goal are incentivized with higher numbers. Unhelpful actions result in a low reward or no reward

Question 13

maximizing total cumulative reward

Answer 13

over time, the agent learns, through trial and error, to map gainful actions to situations

Question 14

Agent

Answer 14

The piece of software you are training is called an agent. It makes decisions in an environment to reach a goal.

Question 15

Environment

Answer 15

The environment is the surrounding area with which the agent interacts.

Question 16

Reward

Answer 16

Feedback is given to an agent for each action it takes in a given state. This feedback is a numerical reward

Question 17

Action

Answer 17

For every state, an agent needs to take an action toward achieving its goal.

Question 18

Episode

Answer 18

An episode represents a period of trial and error when an agent makes decisions and gets feedback from its environment.

Question 19

A reinforcement learning model

Answer 19

an agent learns in an interactive real-time environment by trial and error using feedback from its own actions. Feedback is given in the form of rewards.

Question 20

Training algorithm

Answer 20

Defines the model’s learning objective, which is to maximize total cumulative reward. Different algorithms have different strategies for going about this

Question 21

Soft actor critic (SAC)

Answer 21

embraces exploration and is data-efficient, but can lack stability.

Question 22

Proximal policy optimization (PPO)

Answer 22

is stable but data-hungry

Question 23

An action space

Answer 23

Is the set of all valid actions, or choices, available to an agent as it interacts with an environment.

Question 24

Discrete action space

Answer 24

represents all of an agent’s possible actions for each state in a finite set of steering angle and throttle value combinations.

Question 25

Continuous action space

Answer 25

allows the agent to select an action from a range of values that you define for each state

Question 26

Learning rate

Answer 26

is a hyperparameter that controls how many new experiences are counted in learning at each step. A higher learning rate results in faster training but may reduce the model’s quality.

Question 27

Reward function

Answer 27

Its purpose is to encourage the agent to reach its goal. (Figuring out how to reward which actions is one of your most important jobs.)

Question 28

Key points to remember about reward functions in AWS deep racer

Answer 28

Each state on the grid is assigned a score by your reward function. You incentivize behavior that supports your car’s goal of completing fast laps by giving the highest numbers to the parts of the track on which you want it to drive.

The reward function is the actual code you’ll write to help your agent determine if the action it just took was good or bad, and how good or bad it was.

Question 29

Key points to remember about exploration versus exploitation in AWS deep racer

Answer 29

When a car first starts out, it explores by wandering in random directions. However, the more training an agent gets, the more it learns about an environment. This experience helps it become more confident about the actions it chooses.

Exploitation means the car begins to exploit or use information from previous experiences to help it reach its goal. Different training algorithms utilize exploration and exploitation differently

Question 30

Exploration versus exploitation

Answer 30

An agent should exploit known information from previous experiences to achieve higher cumulative rewards, but it also needs to explore to gain new experiences that can be used in choosing the best actions in the future.

Question 31

Generative AI

Answer 31

is one of the biggest recent advancements in artificial intelligence because of its ability to create new things.

Question 32

discriminative models

Answer 32

A discriminative model aims to answer the question, “If I’m looking at some data, how can I best classify this data or predict a value?” For example, we could use discriminative models to detect if a camera was pointed at a cat.

Question 33

A generative model aims to answer the question

Answer 33

“Have I seen data like this before?” In our image classification example, we might still use a generative model by framing the problem in terms of whether an image with the label “cat” is more similar to data you’ve seen before than an image with the label “no cat.”

However, generative models can be used to support a second use case. The patterns learned in generative models can be used to create brand new examples of data which look similar to the data it seen before.

Question 34

Three popular types of generative models

Answer 34

generative adversarial networks (GANs)
general autoregressive models
transformer-based models.

Question 35

Autoregressive models

Answer 35

Autoregressive convolutional neural networks (AR-CNNs) are used to study systems that evolve over time and assume that the likelihood of some data depends only on what has happened in the past. It’s a useful way of looking at many systems, from weather prediction to stock prediction.

Question 36

Generative adversarial networks (GANs)

Answer 36

Generative adversarial networks (GANs), are a machine learning model format that involves pitting two networks against each other to generate new content. The training algorithm swaps back and forth between training a generator network (responsible for producing new data) and a discriminator network (responsible for measuring how closely the generator network’s data represents the training dataset).

Question 37

Transformer-based models

Answer 37

Transformer-based models are most often used to study data with some sequential structure (such as the sequence of words in a sentence). Transformer-based methods are now a common modern tool for modeling natural language.

Question 38

What are GANs

Answer 38

A GAN is a type of generative machine learning model which pits two neural networks against each other to generate new content: a generator and a discriminator.

A generator is a neural network that learns to create new data resembling the source data on which it was trained.
A discriminator is another neural network trained to differentiate between real and synthetic data.
The generator and the discriminator are trained in alternating cycles. The generator learns to produce more and more realistic data while the discriminator iteratively gets better at learning to differentiate real data from the newly created data.

Question 39

Generator

Answer 39

A neural network that learns to create new data resembling the source data on which it was trained.

Question 40

Discriminator

Answer 40

A neural network trained to differentiate between real and synthetic data.

Question 41

Generator loss

Answer 41

Measures how far the output data deviates from the real data present in the training dataset.

Question 42

Discriminator loss

Answer 42

Evaluates how well the discriminator differentiates between real and fake data.

Question 43

Autoregressive convolutional neural networks

Answer 43

make iterative changes over time to create new data.