Markov Jump Processes

Question 0

Residual holding time for time homogeneous process

Answer 0

P(x(s) = j | x(s) = i, R(s) = w) =

Question 1

What is Markov Jump Process?

Answer 1

A Markov process with

a continuous time set

discreet state space

Question 2

What is distribution of holding time in Markov chain process?

Answer 2

If staying in state 1 transition rate is - mu then holding time T1 would be distributed T1 ~ exp (mu)

Question 3

Find variance of the MLE estimator in Markov chain

Answer 3

Variance can be found with CRLB rule which is -1/second derivative with respect to transition rate.

Question 4

Probability of remaining in a state A for at least 5 years.
Transition rate AA = -sum ( coming out from A transition rates)
Transition rate 0.15

Answer 4

P– (5) = exp (-0.15 *5)

AA

Question 5

Derive an expression for F(i) to be the probability that a person currently in i will never be in state P.

Answer 5

F(A) = mu(AT)/mu(AA) * F(T) + mu(AP) /mu (AA)* F(P) + mu(AD)/mu(AA) * F(D)
Where F(D) = 1 as can never be in P once dead
F(P) = 0 since if currently in state P it is impossible to never visit it

Question 6

Calculate the expected future duration spent in state P for a person currently in state A given the probability that a person in a state A will visit state P is 0.5

Answer 6

If the person will visit state P the timer spent there will be exponentially distributed with parameter lambda which is transitional rate from P (0.2) then mean waiting time in state P will be 1/lambda or 1/0.2 = 5
Then expected future duration spender in state P for a person currently in A : 0.5 * 5

Question 7

What are the parameters of the Markov jump process?

Answer 7

Parameters of the model are the transition rates mu ij between the states where mu ij = lim p ij (h)/ h , i¥j
h - 0
List all transition rates

Question 8

Confidence interval of mu given d and v

Answer 8

Mu = d / v
95% confidence interval use the following formula
Mu +- 1.96*sqroot( mu^2 / d)

Question 9

Irreducible chain definition

Answer 9

Each state can be ultimately reached starting from any other

Question 10

Periodic chain

Answer 10

Chain with period of d is one where return to a given state is only possible in a number of steps that is a a multiple of d

If d =1 then chain is aperiodic

Question 11

Finite chain definition

Answer 11

When a chain has a finite number of spaces

Question 12

If a state space is finite ……

Answer 12

• there is at least one stationary distribution

- but the process may not conform to this distribution in the long term

Question 13

If a chain is finite and irreducible …..

Answer 13

• there is a unique stationary distribution

- but the process may not conform in the long run

Question 14

In chain is finite, irreducible and aperiodic …..

Answer 14

• there is a unique stationary distribution pi
• the process will conform to this distribution in the long term
  lim as n -> ∞ p ij (n) = π j

Question 15

Stationary distribution

Answer 15

If π is a solution satisfying equation π = π P where π j ≥ 0
and Σ π j = 1 then π is a stationary distribution

Question 16

Key assumptions underlying a Markov multiple state model:

Answer 16

• probability of being in state i going to j depends on the current state only no previous information required
• for any two states g and h over a short interval dt
  dtp(t)¹²=μ(t)¹²+o(dt) where t > 0
• μ(t)¹² is constant for 0 < 1

Question 17

When calculating probability of being in state j in n years starting in state i….

Answer 17

The solution would depend on what matrix we are given:
- generator matrix ( with transition rates p ij) then the solution would be
Exp (- λ * t) for probability of staying (hence using holding rate λ )
When jumping from 2 to 3 for example we use μ₂₃/λ₂ where λ₂ is the total force of transition from state 2.
We know that if it is generator matrix the some of transition rate across would be zero

• probability matrix (when the some of probabilities is equal to 1)
  Then probabilities after a few years are found simply rough matrix multiplication!

Question 18

Expected time to reach a subsequent state k formula

Answer 18

is in the tables book next to Kolmogorov forward and backward equations

Question 19

Residual holding time for time homogeneous process

Answer 19

P(x(s) = j | x(s) = i, R(s) = w) =