Week 9

Question 1

The rows in a probability matrix sum to what?

Answer 1

1.

Question 2

What does the splitting of Poisson process theorem say?

Question 3

What is the Poisson process?

Question 4

What is Transient Analysis? How does it work?

Answer 4

It allows you to calculate the n-th step probability matrix for a given problem.

Given that no states are absorbing, the probability of arriving at state i after n steps from state j can be determined

Question 5

What is the Binomial process and what do each of the parameters mean?

Question 6

What is meant by time-homogeneous DTMCs?

Question 7

What are the Chapman Kolmogorov equations and why are they important?

Answer 7

P(m+n) = P(m)P(n), for m,n>=0

P(n) = P^n, for n>= 0

q(u) = q(0)P(n) = q(0)P^n, for n>=0
If you know the first state and the probability matrix, you can determine the probability of arriving at states i,j,k… etc for n steps.

Question 8

What is meant by the “Memoryless Property”?

Answer 8

The future evolution of the process depends on its history only through its present state.

Basically the present state influences the future state but previous states do not.

Question 9

What is a communicating state?

Answer 9

We say i and j communicate, when
P(n)ij >0 for some n>=0 ( i —> j )
P(m)ji>0 for some m>=0 ( j —> i )

If i communicates with j then
j communicates with i

i and j together form a communicating class

Question 10

Draw a transition diagram with the following communicating classes:
{1,2} - closed, {3} - open, {4} - closed

Question 11

When there is only one communicating class the chain is?

Answer 11

Irreducible

Question 12

What is an absorbing state?

Answer 12

A state that you can’t get out of.

Question 12

What is an absorbing state?

Answer 12

A state that you can’t get out of.

Question 13

What is “Discrete Phase-Type Distribution”?

Answer 13

The application potential of this is very powerful!

We apply an absorbing DTMC with states
S= {0, 1, …. , n-1} U {n},
where n is the absorbing state and 0, 1, … , n-1 are transient states.
The matrix P* records the transition probabilities between transient states, and column vector p records the probabilities of absorption to the absorbing state n.

It allows us to determine the time expected until the reaching the absorbing state.

Question 14

What does the geometric distribution describe, and what are it’s parameters?

Question 15

What is the negative binomial distribution?

Question 16

What does it mean if a state is recurrent?

Answer 16

The process will come back to state i. The probability of coming back to i an infinite number of times, is equal to 1.

Question 17

What does it mean if a state is transient?

Question 18

Number of days until it is sunny three days in a row

Answer 18

Negative binomial

Question 19

What distribution would you use to model “Time until it starts raining”?

Answer 19

Exponential suitable if the memoryless property is met.

Otherwise, if you can’t assume independence of events use continuous phase-type distribution.

Question 20

What distribution would you use to model “Time until we see rain three times in a week”?

Answer 20

Erlang

Question 21

What is a phase type-distribution? Give an example of where we would use it?

Question 22

What is lambda?

Answer 22

Rate (unit per time)

Question 23

State and prove the following:

Lemma 1.1(a)-(d) - Properties of the probability measure (complementary event, event that implies another
event, union of two events, union of n events).

Question 24

State and prove the following:

Lemma 2.5 - Properties of the discrete distributions (mean and variance of Binomial, Geometric, Poisson,
Negative Binomial).

Question 25

State and prove the following:

Lemma 2.6 - Physical interpretation of the discrete distributions (Binomial, Geometric, Negative Binomial).

Question 26

State and prove the following:

Lemma 3.2 - Properties of the continuous distributions (cumulative distribution function, mean and variance
of Uniform, Exponential, and Erlang).

Question 27

State and prove the following:

Lemma 3.3 - Memoryless property of the Exponential distribution.

Question 28

State and prove the following:

Theorem 3.3 - Central Limit Theorem.

Question 29

State and prove the following:

Lemma 5.1 (a) - Probability generating function uniquely determines the distribution.

Question 30

State and prove the following:

Theorem 6.1 - Equivalence of two definitions of Poisson Process (PP), in both directions.

Question 31

State and prove the following:

Theorem 7.2 - Chapman-Kolmogorov equations for a DTMC.

Question 32

State and prove the following:

Lemma 8.1(c) - Chapman-Kolmogorov equations for a CTMC.

Question 33

State and prove the following:

Theorem 8.4 - Kolmogorov Differential equations for a CTMC.