LLM Concepts

Question 1

What is a Large Language Model (LLM)?

Answer 1

A deep learning model trained on large corpora of text to understand and generate human language.

Question 2

What architecture do most LLMs use?

Answer 2

The transformer architecture.

Question 3

What is the transformer model?

Answer 3

A model that uses self-attention mechanisms to process sequences in parallel.

Question 4

What is self-attention?

Answer 4

A mechanism where a model learns which parts of a sequence are relevant to each other.

Question 5

What is positional encoding in transformers?

Answer 5

Information added to input tokens to preserve word order.

Question 6

What is a token?

Answer 6

A unit of text, often a word or subword, processed by the model.

Question 7

What is tokenization?

Answer 7

The process of converting text into tokens.

Question 8

What is pretraining in LLMs?

Answer 8

Training the model on large unlabeled text data to learn general language patterns.

Question 9

What is fine-tuning?

Answer 9

Adapting a pretrained model to a specific task with additional labeled data.

Question 10

What is masked language modeling?

Answer 10

A training task where some input tokens are hidden and the model must predict them.

Question 11

What is causal language modeling?

Answer 11

A training task where the model predicts the next token in a sequence.

Question 12

What is GPT?

Answer 12

Generative Pretrained Transformer — a causal language model trained to predict the next word.

Question 13

What is BERT?

Answer 13

Bidirectional Encoder Representations from Transformers — a masked language model.

Question 14

How is GPT different from BERT?

Answer 14

GPT is unidirectional and generative; BERT is bidirectional and better for understanding.

Question 15

What is zero-shot learning?

Answer 15

Making predictions without seeing task-specific examples during training.

Question 16

What is few-shot learning?

Answer 16

Learning a task with only a few examples.

Question 17

What is instruction tuning?

Answer 17

Training LLMs to follow instructions in natural language.

Question 18

What is prompt engineering?

Answer 18

The craft of designing effective input prompts to guide LLM behavior.

Question 19

What is a system prompt?

Answer 19

A special prompt that guides the behavior of an LLM session.

Question 20

What is context window?

Answer 20

The maximum number of tokens an LLM can process at once.

Question 21

What is attention mechanism?

Answer 21

A method that lets models focus on different parts of the input when making predictions.

Question 22

What is temperature in text generation?

Answer 22

A parameter that controls randomness — higher values yield more diverse outputs.

Question 23

What is top-k sampling?

Answer 23

A decoding method that samples from the top k most likely next tokens.

Question 24

What is top-p (nucleus) sampling?

Answer 24

A method that samples from the smallest set of tokens with a cumulative probability > p.

Question 25

What is beam search?

Answer 25

A decoding strategy that keeps multiple candidate sequences at each step.

Question 26

What is hallucination in LLMs?

Answer 26

When a model generates text that is fluent but factually incorrect.

Question 27

What is RLHF?

Answer 27

Reinforcement Learning from Human Feedback — used to align LLMs with human preferences.

Question 28

What is a language model's perplexity?

Answer 28

A measure of how well the model predicts a sample — lower is better.

Question 29

What is an embedding?

Answer 29

A numeric representation of text that captures semantic meaning.

Question 30

What is a vector database?

Answer 30

A database designed to store and search embeddings efficiently.

Question 31

What is retrieval-augmented generation (RAG)?

Answer 31

A technique that combines external knowledge retrieval with generation.

Question 32

What is chain-of-thought prompting?

Answer 32

A method where the model is encouraged to explain its reasoning before answering.

Question 33

What is a decoder-only transformer?

Answer 33

A transformer model that generates output sequentially, like GPT.

Question 34

What is an encoder-only transformer?

Answer 34

A transformer that creates contextual embeddings for input, like BERT.

Question 35

What is an encoder-decoder transformer?

Answer 35

A model architecture used for translation or summarization, like T5.

Question 36

What is T5?

Answer 36

Text-to-Text Transfer Transformer — an encoder-decoder model that treats all tasks as text-to-text.

Question 37

What is parameter count in LLMs?

Answer 37

The number of learnable weights in the model — indicates model size.

Question 38

Why do LLMs need large datasets?

Answer 38

To capture a wide range of language patterns and knowledge.

Question 39

What is data contamination in training?

Answer 39

When test data is accidentally included in training data.

Question 40

What is a safety filter in LLMs?

Answer 40

A mechanism to block harmful or inappropriate outputs.

Question 41

What is the alignment problem?

Answer 41

Ensuring that AI systems behave in accordance with human intent.

Question 42

What is a language model's vocabulary?

Answer 42

The set of tokens it can recognize and generate.

Question 43

What is model distillation?

Answer 43

Compressing a large model into a smaller one that approximates its behavior.

Question 44

What is quantization?

Answer 44

Reducing the precision of model weights to decrease memory usage.

Question 45

What is model pruning?

Answer 45

Removing unnecessary weights or neurons to simplify the model.

Question 46

What is latency in LLMs?

Answer 46

The time it takes to generate a response after receiving input.

Question 47

What is inference in LLMs?

Answer 47

The process of generating predictions using a trained model.

Question 48

What is an API endpoint in LLMs?

Answer 48

A service interface to interact with the model programmatically.

Question 49

What is multi-modal LLM?

Answer 49

A model that processes and generates multiple data types, such as text and images.