Bayesian Statistics

Question 1

Bayes’ theorem

Answer 1

P(A|B) = P(B|A) * P(A) / P(B)

Question 2

For bayesian inference, P(θ|X), define θ and X.

Answer 2

θ is the parameter (what we want to infer from the data, the hypothesis)
X is the data
hence evaluating the probability of a hypothesis given some data.

Question 3

define each part of
P(θ|X) ∝ P(X|θ) * P(θ)

Answer 3

posterior ∝ likelihood * prior

prior = initial belief
updated by the likelihood, also l(θ|X)
posterior is our updated belief AFTER taking the data into account

Question 4

updating posterior with multiple sets of data:
P(θ|X, Y, Z) ∝

Answer 4

P(θ|X, Y, Z) ∝ P(X|θ) * P(Y|θ) * P(Z|θ) * P(θ)

ie. product of individual likelihoods and the prior

Question 5

what does it mean to describe a prior or a likelihood as “normal”?

Answer 5

normal prior: θ is known with some certainty. express as θ0, the believed value, with variance.

normal likelihood function: model uncertainty in a measurement x for the quantity θ

Question 6

what is “precision” 𝜏 ?
how to find precision of the posterior?

Answer 6

defined as the reciprocal of variance.

precision of posterior, 𝜏1 = 𝜏0 + 𝜏
where 𝜏0 is precision of prior, and 𝜏 is precision of likelihood.