AWS AI Practitioner Confusions Flashcards by Varun Pande

ROUGE
Vs
BLEU
Vs
BERTScore
Vs
Perplexity

Compare N-gram matches
Vs
Evaluate Quality(Prcesion and penalizes)
Vs
Semantic similarity(Compare embeddings)
Vs
How confident the model to predict next token(lower is better)

How well did you know this?

Not at all

Perfectly

ROUGE-N
Vs
ROUGE-L

ROUGE-N - This metric primarily assesses the fluency of the text and the extent to which it includes key ideas from the reference. Compare N-gram matches between required vs actual output

ROUGE-L - It is good at evaluating the coherence and order of the narrative in the outputs. Compare the longest sequence of words matche between required vs actual output

How well did you know this?

Not at all

Perfectly

Fine tuned models vs Self trained models

Fine tuning a pre trained model using your data vs training a model from scratch using your data

How well did you know this?

Not at all

Perfectly

Retrieval-augmented generation (RAG)
Vs
Instruction fine-tuning

Supplies domain-relevant data as context to produce responses based on that data.
Vs
Labeled examples and Prompt-response pairs

How well did you know this?

Not at all

Perfectly

Regression
Vs
Classification

Predicting continuous or numerical values based on one or more input variable
Vs
Diagnostic uses which supervised learning technique

How well did you know this?

Not at all

Perfectly

Real Toxicity
Vs
BOLD
Vs
TREX
Vs
WikiText-2

RealToxicityPrompts is a dataset for measuring the degree to which racist, sexist, or otherwise toxic language presents in Pretrained neural language models (LMs).
(Text Generation-Toxicity)
Vs
Bias in Open-ended Language Generation Dataset (BOLD) is a dataset to evaluate fairness in open-ended language generation in English language.
(Text Generation-Toxicity)
Vs
Used for Relation Extraction and Natural Language Generation.
(Text Generation-Accurcy and Robustness)
Vs
Collection of over 100 million tokens extracted from the set of verified Good and Featured articles on Wikipedia
(Text Generation-Robustness)

How well did you know this?

Not at all

Perfectly

Gigaword
Vs
Women’s Ecommerce Clothing Reviews

Gigaword provides headline-generation on a corpus of article pairs consisting of around 4 million articles.
(Text Summarization)
Vs
Dataset revolves around the reviews written by customers
(Text Classification)

How well did you know this?

Not at all

Perfectly

(Question and answer)
BoolQ
Vs
Natural Questions
Vs
Trivia QA

BoolQ is a question answering dataset for yes/no questions containing 15942 examples.
Vs
NaturalQuestions (NQ) contains real user questions issued to Google search, and answers found from Wikipedia by annotators.
Vs
TriviaQA is a reading comprehension dataset containing over 650K question-answer-evidence triples.

How well did you know this?

Not at all

Perfectly

Model tuning method comparison

Prompt Engineering
RAG
Instruction based fine tuning
Domain Adaption fine tuning
Transfer Learning

Prompt Engineering - No model training needed

RAG - Use external knowledge but no FM changes or retraining. Cost for using vector dbs

Instruction based fine tuning - FM is fine tuned with instructions and change the tone of the model. Labelled data and prompt-response pairs

Domain Adaption fine tuning - Domain specific model training. Unlabled data

Transfer Learning - Widely used for image classification

How well did you know this?

Not at all

Perfectly

Temperature
Vs
Top K
Vs
Top P

Creativity of model output
Vs
Most probable response(Number)
Vs
Most likey words(Probability value)

How well did you know this?

Not at all

Perfectly

Amazon Q Business
Vs
Amazon Q Apps
Vs
Amazon Q Developer

Part of Amazon Bedrock with no contol to choose FMs
Vs
Create GenAI apps with use of natural language and no coding
Vs
Generate code and commands related to AWS. Scan code for vulnerabilities. Debugging and Optimization improvements

How well did you know this?

Not at all

Perfectly

Amazon Q Business Lite
Vs
Amazon Q Business Pro

Access to the Q&A feature in Amazon Q Business
Vs
Help you solve problems, generate content, and find insights in data, and Amazon QuickSight, a generative BI assistant to help consume insights.

How well did you know this?

Not at all

Perfectly

GPT
vs
BERT
Vs
RNN
Vs
ResNet
Vs
SVM
Vs
WaveNet
Vs
GAN
Vs
XGBoost

Generate human text or code
Vs
Translation
Vs
Speech recognition
Vs
Image recognition
Vs
Classification & Regression
Vs
Speech Sythesis
Vs
Data augmentation
Vs
Gradient boosting

How well did you know this?

Not at all

Perfectly

KNN
Vs
K-Means

Clustering technique mdoel used in supervised learning
Vs
Clustering technique model used in unsupervised learning

How well did you know this?

Not at all

Perfectly

Underfitting
Vs
Overfitting

High bias and low variance
Vs
Low bias and high variance

How well did you know this?

Not at all

Perfectly

Lexicons
Vs
SSML
Vs
Voice engine
Vs
Speech Mark

Like how to speak and abbreviations
Vs
Adding <break></break>, <whisper></whisper>, etc
Vs
Different types of voice styles
Vs
Helpful for lip synching or highlighting words

How well did you know this?

Not at all

Perfectly

Custom Labels
Vs
Content Moderation
Vs
Amazon A2I

Identify your logo on social media using Amazon Rekognition
Vs
Remove inappropriate content using Amazon Rekognition
Vs
Incorporate human review using Amazon Rekognition

Sagemaker Real time deployment
Vs
Sagemaker Serverless deployment

IaaS (EC2)
Vs
PaaS (Lambda)

ResponsibleAI using various AWS Tools?
Amazon Bedrock, SageMaker Clarify, SageMaker Data Wrangler, SageMaker Model Monitor & A2I

Amazon Bedrock - Guardrails for redacting PII and block undesirable content. Do Human or Automatic Evaluation

SageMaker Clarify - FM evaluation for accuracy, robustness, toxicity and bias detection

SageMaker Data Wrangler - To fix Bias and augment the data

SageMaker Model Monitor - Quality Ananlysis in production

A2I - Human review of ML predictions

Governance using Role manager, Model cards and Dashboard

Interpretability
Vs
Explainability

Degree to which a human can understand the cause of the decision
Vs
Understand the nature and behaviour of the model

ResponsibleAI
Vs
GovernanceAI
Vs
ComplianceAI

Fairness, explainability, interpretibility, transparency, controlability,privacy, safety, robust
Vs
Managing, optimzing and scaling org AI activities with policies, guidelines, risk managment and build public trust
Vs
Complaince to various industry standards for the AI workloads

Data Lifecycles
Vs
Data Logging
Vs
Data Residency
Vs
Data Monitoring
Vs
Data Analysis
Vs
Data Retention
Vs
Data Lineage

Collecting, processing, storage, consumption and archival
Vs
Inputs, outputs, performace metrics and system events
Vs
Where the data is processed and stored
Vs
Data Quality, identifying anomilies and data drift
Vs
Statistical analysis, visualization and exploration
Vs
Regulatory requirements, historical data for training, cost
Vs
Sources of data, licenses and terms of usage or permissions

Threat detection
Vs
Vulnerability Mgmt
Vs
Infrastructue Mgmt

Generating fake content
Vs
Identify software bugs
Vs
Secure cloud computing platform

Accuracy
Vs
Precision
Vs
Recall
Vs
F1-score
Vs
Latency

Ratio of +ve predictions
Vs
Ratio of correct and incorrect +ve predictions
Vs
Ration of correct and incorrect +ve predictions compare to actual
Vs
Average of precision and recall
Vs
Time taken by the model to predict

Posining Vs Jailbreaking Vs Prompt Leaking Vs Exposure Vs Hijacking

Introduction of malicious and bias data during training Vs Gain access to offensive, harmful content using prompts (Is more direct than hijacking) Vs Leaking of prompts and inputs Vs Leaking of sensitive data from training corpus Vs Influencing the output through training data/model manipulation to serve malicious purpose

Logistic Regression Vs Support Vector Machines (SVMs)

Primarily designed for binary classification problems Vs SVMs are effective for classification tasks, especially in high-dimensional spaces

Pretraining Vs Fine Tuning

Uses unlabeled data Vs Uses labeled data

Data drift Vs Hellucination

Input data changes which degrades the output Vs Output appears factutal but misleading and incorrect

Techniques to prevent overfitting Easy Stopping Vs Pruning Vs Regularization Vs Ensembling Vs Data augmentation

Pause the training phase before noise Vs Identify most important feature Vs Apply penalty value to minimal impact feature Vs Combine different ML models predictions Vs Adding small datasets each time of iteration

Shapley values Vs PDP

Shapley values are a local interpretability method Vs Provide a global view of the model’s behavior

Sampling bias Vs Measurement bias Vs Observer bias Vs Confirmation bias

Data used to train the model does not accurately reflect the diversity of the real-world population Vs Inaccuracies in data collection, such as faulty equipment or inconsistent measurement processes Vs Human errors or subjectivity during data analysis or observation Vs Selectively searching for or interpreting information to confirm existing beliefs

Linear regression Vs Document classification Vs Neural networks Vs Decision tree Vs Association rule learning Vs Clustering Which learning techniques?

Supervised Learning Vs Semi-supervised learning Vs Supervised Learning Vs Supervised Learning Vs Unsupervised learning Vs Unsupervised learning

Embedding models Principal component analysis Vs Singular value decomposition Vs Word2Vec Vs BERT

Dimentionality reduction technique Vs Transforms a matrix into a singular matrix Vs Associate words using contunius BOW or Skip-gram Vs Semantic similarity using N-gram matches

AWS Trainium Vs AWS Inferentia

ML chip that AWS purpose-built for deep learning (DL) training Vs ML chip purpose-built by AWS to deliver high-performance inference at a low cost

GenAI Vs ML

Gets features from labels Vs Gets labels from features

Model parallelism Vs Data parallelism

Splitting a model up between multiple instances or nodes Vs Splitting the training set in mini-batches evenly distributed across nodes

Model Parameters Vs Hyperparameters

Internal variables of the model Vs External configurations set before the training process

Multi-modal generative model Vs Multi-modal embedding model

Generate new output Vs Context-based output (Cheaper than generative model)

Training set Vs Validation set Vs Test set

Used to train an algorithm or ML model. The model iteratively uses the data and learns to provide the desired result. Vs Introduces new data to the trained model. You can use a validation set to periodically measure model performance as training is happening, and also tune any hyperparameters of the model. However, validation datasets are optional. Vs Used on the final trained model to assess its performance on unseen data. This helps determine how well the model generalizes.

SHapley Additive exPlanations Vs Differential privacy Vs Adversarial debiasing Vs Fairness-aware preprocessing

Explain model predictions and identify feature importance Vs Protects individual privacy Vs Mitigation method but is typically applied during or after training Vs Proactive responsible AI strategy that helps reduce bias before the model is trained

Diffusion Model Vs GAN

Diffusion models have gained popularity over GANs due to their ability to generate high-quality images with superior fine-grained control. They are slower than GAN

Recurrent Neural Network (RNN) Vs Generative Adversarial Network (GAN) Vs Transformer-based vision-language model

Sequential data processing, such as text or time-series analysis Vs Image generation, style transfer, and upscaling Vs Generating text descriptions from images