Lecture 3b Flashcards
(12 cards)
Mutually Exclusive Definition:
The system must only be in ONE state at EACH time
Exhaustive
the system must be in ONE state at ALL times
In a discrete state what does X(3) = 5 mean?
the system occupies state number 5 at time T_3
What is the goal of the Markov process?
Compute the probability that the system is in a give state at a given time, for all possible states and times
What is the difference between a Stochastic Process and a Markov process?
Stochastic Process: the probability of a future state usually depends on in entire history
Markov process: the probability of a future state ONLY depends on the PRESENT state!
–> the system has no Memory
What is the transition probability:
Probability that the system in state “i” at time T_m moves to state j at time t_n
Properties:
- are larger or equal to zero
- they must sum to One.
3.
What does the transition probability depend on?
It depends on the time interval not on the individual time -
What are the main premises of the Markov process:
- Process is Stochastic
- process is homogenous in time
- states are mutually exclusive and exhaustive
- the process is memoryless and only depends on the current state of the system
What is a recurring state?
If a system starting at state i will SURLY return to this state i at some point
Pi “NOT” = 0
What is a Transient state?
a State i is transient if the system starting at such state has a finite probability of never returning to it
Pi = 0
Can we have a Markov process where all states are transients? Why / Why not?
We cannot because eventually It will leave the transient state and we MUST have a steady state
What is an absorbing state?
A state is absorbing if we cannot leave it once it enters
Pii = 1
