Introduction to Continuous time markov processes Flashcards
(15 cards)
What is the key difference between discrete-time and continuous-time Markov processes?
In discrete-time Markov processes, time is measured in discrete steps, while in continuous-time Markov processes, time is continuous and the process can change states at any instant.
What is the definition of a continuous-time Markov process?
A continuous-time Markov process is a stochastic process where the future state depends only on the current state, not on the past states, and the process is defined for all t ≥ 0.
What is the transition probability for continuous-time Markov processes?
The transition probability p_ij(t) is the probability that the process is in state j at time t given that it started in state i at time 0.
How is the transition matrix P(t) for continuous-time Markov processes structured?
The transition matrix P(t) contains the transition probabilities p_ij(t) for all states i and j, where the elements of the matrix must satisfy p_ij(t) ≥ 0 and each row sums to 1.
What happens when t = 0 in the transition matrix for continuous-time Markov processes?
When t = 0, the transition matrix P(0) is the identity matrix I, meaning the process is certain to stay in its initial state.
What is the exponential distribution and why is it important for continuous-time Markov processes?
The exponential distribution is memoryless and fundamental for continuous-time Markov processes, as it models the time between events in a Poisson process.
What is the ‘lack-of-memory’ property of the exponential distribution?
The lack-of-memory property means that the probability of an event occurring in the future is independent of how much time has already passed.
What is the mathematical form of the lack-of-memory property for the exponential distribution?
For an exponential distribution with parameter λ, the probability that the event occurs after an additional time s, given that t time has already passed, is P(X > s+t | X > t) = P(X > s).
What is an example of applying the lack-of-memory property?
In a bank with exponentially distributed service times, if you wait for t minutes, the remaining service time is still exponentially distributed with the same parameter λ.
What is the probability that you will be the last to leave in a queue with two clerks?
The probability is 1/2, because the service times of the two customers are independent and exponentially distributed, and by the lack-of-memory property, the two remaining times are equally likely.
What happens when you take the minimum of several independent exponential random variables?
The minimum of several independent exponential random variables is also exponentially distributed, with a rate parameter equal to the sum of the individual rates.
What is the probability that Yk is the minimum of several independent exponential random variables?
The probability that Yk is the minimum is λk / (λ1 + … + λn), where λk is the rate parameter of Yk.
What does the ‘o(h)’ notation mean?
‘o(h)’ means that the function f(h) tends to 0 faster than h as h → 0. It represents a small-o notation used to describe the asymptotic behavior of functions.
In Example 6.2, what is the probability that X ∈ (t, t+h] given X > t?
For an exponential random variable X with parameter λ, the probability that X ∈ (t, t+h] given X > t is λh + o(h).
What is the key feature of exponential random variables in continuous-time Markov processes?
Exponential random variables are memoryless, which means the future behavior of the process is independent of the past.
