Probabilistic recursion

58 Views Asked by At

I have the following recursion:

$$p_{t+1} = \begin{cases} 1 \text{ with probability } 1-p_t \\ \alpha p_t \text{ with probability } p_t \end{cases}$$

for $0<\alpha<1$. Numerical simulations show that $\mathbb{E}[p_t]$ converges to a steady-state value as $t\rightarrow \infty$, irrespective of the initial condition, only dependent on $\alpha$. I'm not sure how to approach solving it in closed-form. It seems like it should be possible.

I have tried solving it by imposing a self-consistent condition, i.e.

$$\mathbb{E}[p_{\infty}] = \begin{cases} 1 \text{ with probability } 1-\mathbb{E}[p_{\infty}] \\ \alpha \mathbb{E}[p_{\infty}] \text{ with probability } \mathbb{E}[p_{\infty}] \end{cases}$$

I'm not sure if this or the next step is valid, I can neither justify it nor state exactly what's going wrong (but something is wrong, as the answer I get does not match my simuluations). Setting $\mathbb{E}[p_{\infty}] = 1\cdot(1-\mathbb{E}[p_{\infty}]) + \alpha \mathbb{E}[p_{\infty}] \cdot \mathbb{E}[p_{\infty}]$ and solving for $\mathbb{E}[p_{\infty}]$ gives a value which is different from simulations.

Can someone provide a solution or a resource for solving such probabilistic recurrences, or let me know where I'm going wrong and how to proceed?

1

There are 1 best solutions below

0
On

An observation that might prove fruitful for you, too much for a comment, but likely not the answer you're looking for. Anyway ...

It seems your interest is in the long-time behavior of $p_t$. When your process starts, there is considerable flexibility in how $p_t$ evolves. However, that changes the first time you hit the "set $p_t = 1$" situation. The next step after will necessarily yield $p_{t+1} = \alpha$, and the one following $p_{t+2}$ being set either to $\alpha^2$ or back to 1, etc. So after hitting that $p_t = 1$ reset button, all the values of $p_{t'}$ you generate are discrete and of the form $1, \alpha, \alpha^2 \alpha^3, \dots$.

This helps you understand the later process, which you might imagine as having states $S_0, S_1, S_2, S_3, \dots$. State $S_k$ has value $\alpha^k$, and you can write your process in terms of the transition matrix \begin{equation} T = \begin{bmatrix} 0 & 1 & 0 & 0 & 0 & 0 & \cdots \\ 1 - \alpha & 0 & \alpha & 0 & 0 & 0 & \cdots \\ 1 - \alpha^2 & 0 & 0 & \alpha^2 & 0 & 0 & \cdots \\ 1 - \alpha^3 & 0 & 0 & 0 & \alpha^3 & 0 & \cdots \\ 1 - \alpha^4 & 0 & 0 & 0 & 0 & \alpha^4 & \cdots \\ 1 - \alpha^5 & 0 & 0 & 0 & 0 & 0 & \cdots \\ \vdots & \vdots& \vdots & \vdots & \vdots & \vdots & \end{bmatrix} \end{equation} The Perron eigenvector $[\pi_0, \pi_1, \pi_2, \dots]$ of $T$ gives you the distribution of the steady state, and the expected value of $p_t$ tending to $p_\infty = \pi_0 + \pi_1\alpha + \pi_2\alpha^2 + \cdots$.