Classifying discrete time markov processes Flashcards
(17 cards)
What is an irreducible class of states?
An irreducible class is a set of states where every state can reach every other state within the class.
What does it mean for a Markov chain to be irreducible?
A Markov chain is irreducible if there is only one irreducible class of states.
What is the definition of recurrence and transience?
A state is recurrent if the probability of returning to it is 1; it is transient if the probability of returning to it is less than 1.
What is the first passage time?
The first passage time is the number of steps it takes for a Markov chain to return to a state, given it started in that state.
What is the difference between positive recurrence and null recurrence?
Positive recurrence means the expected return time is finite, while null recurrence means the expected return time is infinite.
What is periodicity in a Markov chain?
Periodicity refers to the greatest common divisor of the set of times at which a state can return to itself.
What is a closed class in a Markov chain?
A closed class is a set of intercommunicating states where once the chain enters, it cannot leave.
What is an absorbing state?
An absorbing state is a state that forms a closed class on its own, meaning once the chain enters this state, it stays there forever.
What are the class properties in a Markov chain?
Class properties include recurrence, transience, null recurrence, positive recurrence, and periodicity.
What is the theorem about irreducible finite Markov chains and null recurrence?
A finite irreducible Markov chain cannot be null recurrent.
What does the theorem about finite Markov chains and transience state?
It is not possible for all states in a finite state space Markov chain to be transient.
What is the decomposition of states in a Markov chain?
The state space of a Markov chain can be decomposed into transient states and irreducible closed classes of recurrent states.
What happens in a finite state space Markov chain?
In a finite state space Markov chain, there must be at least one recurrent state, and there are no null recurrent states.
What is the relationship between null recurrent and positive recurrent states?
Null recurrence and positive recurrence are class properties; if one state in an irreducible class is null recurrent, all states in that class are null recurrent.
What is the consequence of a finite closed irreducible class?
A finite closed irreducible class must be positive recurrent.
What does the theorem on the irreducibility of recurrent states state?
If a class of recurrent states is irreducible, then it must be closed.
What happens when the state space is finite in a Markov chain?
If the state space is finite, then at least one state must be visited infinitely often, and the chain must eventually enter a recurrent state.
