Calculating cumulative Markov Chain outcomes

Question

I have a Markov process, with 2 possible states (1 or 0) and a transition matrix P. State at time t=n is determined by x⁰*Pⁿ. As n goes to infinity, xⁿ goes to the steady state vector, q = [q1 q2].

I am modeling a situation with strong positive autocorrelation, so P looks something like:

⎡ 0.99 0.01 ⎤
⎣ 0.005 0.995⎦ ,

where Pr(Success|previous Success) = Pr(S|S) = 0.99 and Pr(Success| previous Failure) = Pr(S|F) = 0.005. Thus, this chain creates long strings of successes or failures.

I want to model Pr(Success) on a fine scale, say, every 3 seconds. I have empirical evidence for Pr(at least 1 success in a 10 minute period). So what I need is an equation relating Pr(at least 1 success in 10 minutes) to Pr(S|S) and Pr(S|F).

I was able to do this in a simulated setting, and it turns out that the values for P above produce a Pr(at least 1 success in 10 minutes) ~ 0.6. I would like to hold Pr(S|S) constant and adjust Pr(S|F) (or q, since that has the same effect) to get the Pr(at least 1 success in 10 minutes) I need.

What equation will relate Pr(at least 1 success in 10 minutes) to Pr(S|S) and Pr(S|F)? I used simulations and regression to get the line Pr(at least 1 success in 10 minutes) = 1-exp(k*q1(P(S|S)-1)), which is close, but it is not quite right.

Thanks in advance!

Answer 1

$\DeclareMathOperator{\Pr}{Pr}$I'm not sure whether the time $t$ is measured in seconds or minutes. I'll assume minutes; if it's seconds, please replace $10$ with $600$ everywhere below.

Here, as often, it's easiest to calculate "the probability of at least one" by calculating "the probability of none" and then taking the complement.

Let $X_t$ denote the state of the process at time $t$ and suppose for simplicity that we start at time $0$. We want to calculate $\Pr(X_1, \dots, X_{10} = F)$. This equals $$\Pr(X_1, \dots, X_{10} = F \mid X_0 = F)\Pr(X_0 = F) +\Pr(X_1, \dots, X_{10} = F \mid X_0 = S)\Pr(X_0 = S),$$ which equals $$\Pr(F \mid F)^{10} \Pr(X_0 = F) + \Pr(F \mid F)^9 \Pr(F\mid S) \Pr(X_ 0 = S).$$ Observing that $\Pr(S \mid F) = 1 - \Pr(F \mid F)$ will give you what you wanted.