Continuous-time Markov chains

Question 1

Describe a birth process

Answer 1

N_0 >= 0 and s N_s < N_t
Single arrival property
Conditionally independent increments

pg 110

Question 2

What is a stochastic semigroup?

Answer 2

P_t = (p_ij(t) ) satisfies

1. P_0 = I
2. P_t is stochastic
3. P_{s+t} = P_s P_t (CK equations)

Question 3

When is the semigroup P_t called standard?

Answer 3

lim (t↓0)Pt = I (= P0)

Question 4

Define holding times of a continuous time Markov chain. What distribution do they follow?

Answer 4

H|i = inf{s ≥ 0 : Xt+s =/= i | Xt = i} = distr. = inf{s ≥ 0 : Xs =/= i | X0 = i}
They follow an exponential distribution (pg100)

Question 5

State the Kolmogorov forward and backward equations

Answer 5

a continuous-time Markov chain with
stochastic semigroup {Pt} and generator G satisfies
forward: P’_t = P_tG
backward: P’_t = GP_t

Question 6

Define the generator of a Markov chain

Answer 6

The generator G = (gij )i,j∈E of the Markov chain with stochastic semigroup Pt is defined as the card(E) × card(E)-matrix given by
G := limδ↓0 1/δ[P_δ − I] = limδ↓0 1/δ[P_δ − P_0],
That is, P_t is differentiable at t=0

Question 7

Write P_t in matrix exponential form

Answer 7

P_t = sum_{n=0}^inf (t^n / n!) G^n = e^{tG}

note: In a continuous time Markov chain, if pij (t) > 0, for some t > 0, then pij (t) > 0 for all t > 0.

pg 105

Question 8

What is an irreducible Markov chain?

Answer 8

One where the transition probability matrix P is s.t. for any i, j in E we have p_ij(n) > 0 for some n

Question 9

Define the limiting distribution of a continuous time markov chain

Answer 9

A distribution π is the limiting distribution of a continuous-time Markov chain if, for all states i, j ∈ E, we have
lim t→∞ pij (t) = πj

Question 10

When do we have π = πP_t for all t ≥ 0 ?

Answer 10

Subject to regularity conditions, we have π = πPt for all t ≥ 0 if and only if πG = 0