4) Long run behaviour of discrete markov chains Flashcards
(19 cards)
What is the long-run behaviour of a Markov chain?
The long-run behaviour refers to the distribution of states p(n) as n → ∞. We want to know if it converges and what the limit is.
What is an invariant distribution?
An invariant distribution π is a probability vector that satisfies π = πP, meaning if the chain starts in this distribution, it remains in it at all times.
What happens if the initial distribution is the invariant distribution?
If the initial distribution is the invariant distribution, then the distribution of states p(n) remains the same (π) for all n.
What are the consequences of having an invariant distribution?
The distribution of states at any time n remains the same if the initial distribution is the invariant distribution. The system will be in equilibrium.
How do we find an invariant distribution?
To find an invariant distribution, we solve the equation π = πP, which results in a set of simultaneous linear equations.
How many invariant distributions can there be?
There can be no solution, exactly one solution, or an infinite number of solutions, depending on the structure of the chain.
What is the relationship between irreducibility and invariant distributions?
If the chain is irreducible, an invariant distribution exists and is unique if all states are positive recurrent.
What is an equilibrium distribution?
An equilibrium distribution is the limiting distribution of the Markov chain, where p(n) → π as n → ∞, regardless of the initial distribution.
How is an equilibrium distribution related to the invariant distribution?
An equilibrium distribution is an invariant distribution, but an invariant distribution is not necessarily an equilibrium distribution.
What happens when a Markov chain has an equilibrium distribution?
When a Markov chain has an equilibrium distribution, the chain will converge to that distribution over time, regardless of the starting state.
What is the difference between invariant and equilibrium distributions?
An invariant distribution is one that remains unchanged over time if the chain starts in it, while an equilibrium distribution describes the limiting distribution as n → ∞.
What is the significance of ergodicity in equilibrium distribution?
Ergodicity (positive recurrence and aperiodicity) ensures the existence of an equilibrium distribution, as long as the chain will eventually be absorbed into a single ergodic class.
What is the criterion for the existence of an equilibrium distribution?
An equilibrium distribution exists if and only if the Markov chain has a single ergodic class and is guaranteed to eventually enter it from any starting point.
How does an irreducible ergodic Markov chain behave in terms of equilibrium?
An irreducible ergodic Markov chain has a unique equilibrium distribution, and the chain will eventually converge to this distribution.
What is the relationship between the equilibrium distribution and mean recurrence time?
For an irreducible, aperiodic chain, the equilibrium distribution π_j satisfies π_j = 1/μ_j, where μ_j is the mean recurrence time of state j.
When does an equilibrium distribution not exist?
An equilibrium distribution does not exist if there are multiple closed classes or if all states are transient.
What happens in a Markov chain with multiple closed classes?
If there are multiple closed classes, no equilibrium distribution exists because the long-term behaviour depends on the initial distribution.
What is the key criterion for an equilibrium distribution to exist?
For an equilibrium distribution to exist, there must be exactly one closed class, and the chain must eventually be absorbed into this class.
What is the summary of when equilibrium distributions exist?
If there are multiple closed classes or all states are transient, no equilibrium distribution exists. If there is exactly one closed class and it is ergodic, then an equilibrium distribution exists.
