Stochasic Processes

Question 1

How to find expected time to reach a given state

Answer 1

Create simultaneous equations showing the expected time to go from a given state to the target state qi, multiplying the probability of every path by the time to get there after that. Remember to add one to the terms to count for the time to make that move ie (1+q2) like terms.

Question 2

Equation for distribution after n steps

Answer 2

Xn = Xo P^n

Post multiplying by transition matrix is standard

Question 3

Equation to help find limiting distribution

Answer 3

X inf = Xinf P

     = X0 Pinf 

Limiting distribution is Xinf and after denoted as π

Question 4

What is the period of a state

Answer 4

The greatest common divisor of the set

Question 5

What makes a state aperiodic

Answer 5

It has a period 1

Period is the largest common divisor of the number of steps it takes to go from one state and back to itself

A state is