Bayesian ML Flashcards
(28 cards)
What is the main goal of Bayesian machine learning?
To model uncertainty in parameters, data, and models consistently.
What does a Bayesian model treat as uncertain?
Parameters, predictions, hypotheses, and even model structure.
What key idea underlies the Bayesian approach to knowledge?
Beliefs are updated through evidence using probability.
What separates inference from decision in Bayesian ML?
Inference models belief given data; decision uses that belief to act.
What is the purpose of the prior in Bayes’ theorem?
It represents our belief before seeing data.
What is the purpose of the likelihood in Bayes’ theorem?
It measures how well parameters explain the data.
What is the posterior in Bayesian inference?
The updated belief about parameters after seeing data.
What is the marginal likelihood (evidence) in Bayes’ rule?
The normalising constant that ensures a proper posterior.
What principle does Bayesian inference naturally implement?
Occam’s Razor—preferring simpler models that still explain data.
What does marginalisation allow in Bayesian inference?
Averaging over uncertainty and discarding irrelevant variables.
What does it mean to say Bayesian inference is probabilistic?
All predictions are distributions, not point estimates.
Why is Bayesian ML good for small datasets?
It leverages prior knowledge and doesn’t overfit easily.
What is one advantage of Bayesian methods in streaming data?
They allow incremental updating as new data arrives.
What is Occam’s Razor?
A principle that favours simpler explanations when multiple are possible.
How does Bayesian ML use Occam’s Razor?
By penalising overly complex models via the marginal likelihood.
What real-world problem was used to show Bayesian sparsity?
Classifying leukemia with gene expression data.
What is a key benefit of Bayesian models in high dimensions?
They automatically ignore irrelevant features.
What does a sparse prior encourage in Bayesian inference?
Solutions that use only the most relevant components.
What is an inverse problem in ML?
Inferring causes from observed effects, often under uncertainty.
Why is Bayesian inference useful for inverse problems?
It models uncertainty and finds sparse, plausible explanations.
What method was used to infer radioactive sources in the slides?
Bayesian inversion using sparse gamma ray data.
What is a key downside of Bayesian inference?
Computations are often intractable and require approximation.
What is an example of a Bayesian approximation method?
Markov Chain Monte Carlo (MCMC) or variational inference.
Who first proposed Bayesian inference?
Thomas Bayes, in 1763.
