week 6 - language

Question 1

Computational Linguistics

Answer 1

The science of how language works using math + logic

Question 2

NLP (Natural Language Processing)

Answer 2

AI tools for working with language (like translation, chatbots)

Question 3

Chomsky’s Generative Grammar

Answer 3

Early formal models of language structure.

Question 4

eliza

Answer 4

One of the first chatbot programs; mimicked a psychotherapist but lacked true understanding.

Question 5

Neural Networks

Answer 5

AI models inspired by how neurons work.

Question 6

Word Embeddings

Answer 6

words represented as vectors are positioned in a multidimensional space

Examples: “dog” and “cat” vectors placed closely together.

Question 7

transformers

Answer 7

sequence to sequence model based on deep neural networks (deep learning) with multi head attention

ie, used in translation

input (encoder): English
output (decoder): french

Question 8

Self-attention mechanism:

Answer 8

Model looks at all words in a sequence, not just the last ones.

Question 9

The Transformer consists of two main parts:

Answer 9

encoder
embedding
decoder

Question 10

encoder

Answer 10

takes the input sequence and maps it into an embedding

Question 11

embedding

Answer 11

a n-dimensional vector representing the sequence

Question 12

decoder

Answer 12

takes the embedding and turns it into the output sequence

Question 13

self attention

Answer 13

allows the transformer to look at the other positions in the input sequence for clues that can help lead to a better encoding of the world

Question 14

self attention diff than previous models

Answer 14

can attend at all the words in the sequence, not just the last ones

Question 15

Applications of Transformers:

Answer 15

Machine translation

Summarization

doc generation

Named Entity Recognition (NER)

Biological sequence analysis

Computer vision

Protein folding

Code generation

Question 16

language model

Answer 16

Predicts the probability of the next word in a sequence.

Question 17

language model ex

Answer 17

GPT, PaLM, LLaMA, Bard, Claude

Question 18

language model training

Answer 18

Pre-trained on massive datasets (hundreds of billions of tokens).

Question 19

GPT-style Models

Answer 19

Decoder-only models.

Predict one word at a time.

Example: ChatGPT (Generative Pre-trained Transformer).

Question 20

Prompt Engineering

Answer 20

Crafting prompts to guide LLM behavior.

Question 21

RLHF (Reinforcement Learning with Human Feedback)

Answer 21

Fine-tuning models using human evaluations.

Question 22

in context learning

Answer 22

an example to teach the llm how to respond (one shot)

art of asking the right question to get the best output from an llm - enables direct interaction with the llm using only plain lang prompts

Question 23

llm problems

Answer 23

performance disparities
bias
misinformation
privacy and security
ethical concerns
environmental concerns

Question 24

toxicity

Answer 24

anything that is rude

chatbox could reply with a toxic response

Question 25

language shift

Answer 25

main driver of lanuage endangerment and extinction speakers switch from a native to dominant national language driven by economical factors and marginization of indigenous communities

Question 26

role of ai in language decline

Answer 26

reinforcing english

Question 27

why is english dominant language

Answer 27

global reach data availability research focus

Question 28

BERT

Answer 28

computer model that helps machines understand the meaning of words in a sentence by looking at the words before and after them. It’s used to improve tasks like answering questions, translating language, and understanding text better.

Question 29

What does NLP (Natural Language Processing) focus on? A) Building theoretical models of language B) Practical applications like translation and chatbots C) Creating hardware for AI D) Writing poetry

Answer 29

b

Question 30

What key paper introduced the Transformer model? A) "Attention is All You Need" B) "Generative Grammars" C) "The History Manifesto" D) "Computational Linguistics"

Answer 30

a

Question 31

Which of the following best describes word embeddings? A) Words as vectors based on their context B) Hard-coded grammar rules C) A speech recognition method D) Simple keyword matching

Answer 31

a

Question 32

What is a Language Model’s primary function? A) To summarize texts B) To translate languages C) To predict the next word in a sequence D) To scan images

Answer 32

c

Question 33

What does RLHF stand for in AI training? A) Reinforcement Learning with Human Feedback B) Recursive Language Handling Framework C) Real-time Language Heuristic Function D) Random Learning High Fidelity

Answer 33

a

Question 34

. Which of the following is NOT a risk associated with LLMs? A) Bias and unfairness B) Gender stereotypes C) Toxicity D) Unlimited computing power

Answer 34

d

Question 35

What does ‘Prompt Engineering’ involve? A) Writing computer code to train LLMs B) Creating the best instructions or questions to guide AI responses C) Designing hardware for AI systems D) Debugging AI software

Answer 35

b

Question 36

What is ‘Emergent Abilities’ in Large Language Models? A) Abilities that appear only after specific training for each task B) Abilities that develop spontaneously after a certain model size C) A type of bug in AI models D) Manual programming of new skills

Answer 36

b

Question 37

. Which of these is a core component of the Transformer architecture? A) Recurrent loops B) Self-attention mechanism C) Decision trees D) Support vector machines

Answer 37

b

Question 38

Why is the environmental impact of LLMs a concern? A) They require a lot of data storage B) Training and running models use high energy and produce carbon emissions C) They cannot run on solar power D) They produce toxic output

Answer 38

b