Vocabulary Bank Flashcards
(31 cards)
Backpropagation through time (BPTT)
gradient-based technique for training RNNs by
unfolding them in time and applying backpropagation to change all the parameters in the RNN
Batch size
the number of training examples utilized in one forward/backward pass through the
network, before the loss and subsequently gradients are calculated
Bag-of-words
A text representation method in NLP where a document is represented as a
vector of word frequencies, ignoring grammar and word order.
Biases
Systematic errors in a dataset that can lead to unfair outcomes in a model
Types of Biases
Confirmation
Historical
Labeling
Linguistic
Sampling
Selection
Dataset
A collection of data used for training or evaluating machine learning models
Deep learning
A subset of machine learning involving neural networks with many layers that
can learn representations of data
Graphical processing unit (GPU)
A specialized hardware component designed to handle and
accelerate parallel processing tasks, particularly effective for rendering graphics and training
deep learning models by performing simultaneous computations across multiple cores
Hyperparameter tuning
The process of optimizing the parameters that govern the training
process of machine learning models to improve performance
Large language model (LLM)
A type of AI model trained on vast amounts of text data to
understand and generate human-like text
Latency
The delay between the input to a system and the corresponding output.
Learning rate
controls the size of the steps the model takes when updating its parameters
during training - if the learning rate is increased, weights and biases of the network are updated
more significantly in each iteration
Long short-term memory (LSTM)
A type of RNN designed to remember information for long
periods and mitigate the vanishing gradient problem
Long-term dependency
refers to the challenge in sequence models, like Recurrent Neural
Networks (RNNs), of capturing and utilizing information from earlier in the input sequence to
make accurate predictions at later time steps
Loss function
A function that measures the difference between the predicted output and the
actual output, guiding model training.
Memory cell state
In LSTM networks, the cell state carries long-term memory through the
network, allowing it to retain information across time steps
Natural language processing
The field of AI focused on the interaction between computers
and human language
Discourse integration
Understanding and maintaining coherence across multiple
sentences or turns in conversation
Lexical analysis
The process of examining the structure of words.
Pragmatic analysis
Understanding language in context, including the intended
meaning and implications
Semantic analysis
The process of understanding the meaning of words and sentences
Syntactical analysis (parsing)
Analyzing the grammatical structure of sentences
Natural language understanding (NLU)
A modular set of systems that sequentially process
text input to better represent their meaning before they are input into a neural network such as a
transformer NN or LSTM
Pre-processing
The process of cleaning and preparing raw data for analysis or model training
