L8 - Recurrent Neural Networks

Question 1

What is a Recurrent Neural Network (RNN)?

Answer 1

A Recurrent Neural Network (RNN) is a type of neural network designed to handle sequential data — data where the order of elements matters. This makes RNNs especially useful for tasks like:

• Natural language processing (e.g., text generation, translation)
• Time series prediction (e.g., stock prices, weather forecasting)
• Speech recognition
• Music generation

Question 2

How do RNNs differ from standard Neural Networks?

Answer 2

Unlike traditional neural networks, RNNs have loops in them, allowing information to persist. They process input one element at a time while maintaining a hidden state that carries information from previous steps.

At each time step:

The RNN takes an input (e.g., a word or a data point).
It updates its hidden state based on the current input and the previous hidden state.
It optionally produces an output.
This structure allows RNNs to have a kind of memory, making them suitable for tasks where context is important.

Question 3

RNNs use Long Short Term Memory what does this mean?

Answer 3

LSTM
1. They forget the irrelevant parts of the previous state
2. Store the relevant new information into the cell state
3. Selectively update the cell state values
4. Output some information to the next step.