I have a stochastic process with two states: A and B (pic here). The transition probability is dependent on the number of all past occurrences of the state B.
- The transition probability from A to B ($\beta^t$): $\beta^t = 0.25*[Count(B)^t+1]^{-2}$
- The transition probability from B to A ($\gamma^t$): $\gamma^t = 1 - 0.75*e^{-2*Count(B)^t}$
- The transition probability from A to A is $1-\beta^t$
- The transition probability from B to B is $1-\gamma^t$
where $Count(B)^t$ is the count of all past occurrences of B from $t=0$ till now (time $t$). $\beta^t$ and $\gamma^t$ can determine the $t+1$ state.
I was wondering:
- What is the infinite-time status of the stochastic process? Can we characterize $\lim_{t \to \infty}\beta^t$, $\lim_{t \to \infty}\gamma^t$, $\lim_{t \to \infty}Count(B)^t$?
[My intuition is that $\lim_{t \to \infty}Count(B)^t \to \infty$. But plugging in the equation of $\beta^t$, $\lim_{t \to \infty}\beta^t=0$, meaning there will be no "B", which contradicts with $\lim_{t \to \infty}Count(B)^t \to \infty$. What is right and what is wrong here? What is the infinite-time status?]
- More broadly, is the setup of this stochastic process "valid"? Is there any paper / book that has mentioned this or similar kind of process? (I cannot find any...)
Thanks a lot for your help!
Think of it this way: if you're currently at A, the time to wait to get to B again is a geometric random variable with a success probability that decreases each time you go to B. This probability is always positive, so with probability 1 you succeed eventually, and then return to A eventually, etc. The time to wait grows, but it never becomes infinite. Nonetheless that growth means that the long-term distribution is concentrated at A; excursions to B become less and less frequent and also more and more brief because of the decay of $\beta$ and the growth of $\gamma$.
In any case, such a thing is well-defined but as far as I know such processes aren't very well-studied (because they have arbitrarily long memory).