# ML Models Flashcards

Cons of Logistic Regression

- Non-linear problems can’t be solved with LR (only produces a linear decision boundary).
- Can’t capture feature interactions when the value of one feature influences the value of another (between user, ad, and publisher for instance)

Pros of Logistic Regression

Pros:

- Easy to implement

- Easy to train

- Fast inference

- Interpretable

- Often useful as a baseline model

Logistic Regression

- Models the probability of a binary outcome using a weighted linear combination of features.
- Work well when the data is linearly separable
- Not a good choice for ad click prediction

Gradient-boosted decision trees

Pros

- Interpretable and easy to understand

- Can be used for feature selection (importance) and feature extraction

Cons

- Inefficient for continual learning. Not designed to be fine-tuned with new data. Usually need to retrain the model from scratch.

Two-tower neural network

- Generate user embeddings for user features
- Generate ad embeddings for ad features
- The similarity between the user and ad embeddings is used to calculate relevance

Challenges of ad click prediction

- Feature space is large and sparse. most features are filled with zeros.
- Difficult to capture pairwise interactions
- Continuous retraining

Deep & Cross Network

- Can replace manual feature cross method
- Deep network: Learns complex generalizable features using DNN arch
- Cross network: Automatically captures feature interactions and learns good feature crosses

Factorization machines

- Efficiently captures pairwise interactions between features
- Improves logistic regression
- Useful for ad click prediction

How do factorization machines work?

Learns an embedding vector for each feature. The interaction between two features is the dot product of their embeddings

Support Vector Machines

Kind of like logistic regression in multi-dimensional space

- Find a shape in n-dim space that classifies data points

What is Learn To Rank?

- Supervised machine learning to solve ranking problems
- Given a query and a list of items, determine the optimal ordering of the items from most relevant to least relevant

What are the types of Learn to Rank?

- Pointwise
- Pairwise
- Listwise

Point-wise Learn to Rank

- The score of each item is predicted independently of the other items
- The final ranking is achieved by sorting the predicted relevance scores

Pair-wise Learn to Rank

- Given a query and two items, predicts which item is more relevant to the query
- Examples: RankNet, LambdaRank, LambdaMART

List-wise Learn to Rank

- Given a query and a list of items, predict the optimal ordering of an entire list
- Examples: SoftRank, ListNet, and AdaRank