Markov Process with time varying transition probabilities.

172 Views Asked by At

I am interested in studying the evolution of a variable $\alpha_t\in [0,1]$ governed by the following stochastic dynamical system:

$$ \begin{cases} \alpha_0\in [0,1]\\ \alpha_{t+1}=\frac{t+1}{t+2}\alpha_{t} + \frac{1}{t+2}W_t\\ W_t\sim Bernoulli(P(\alpha_{t})) \end{cases} $$ where $P:[0,1]\to[0,1]$ is a continuous decreasing function such that $P(0.5)=0.5$ and there exist $0<a<1/2<b<1$ such that $P(x)=1$ for $x\in [0,a]$ and $P(x)=0$ for $x\in [b,1]$.

A qualitative study of the system reveals that $\alpha_t$ will be eventually bound in $[a,b]$. At the same time, I would like to have a more precise prediction about the asymptotic behavior of $\alpha_t$. Running all sorts of simulation, one has that asymptotically $$\alpha_t\to 1/2$$ This is somewhat intuitive because, for a very large $T$ we have $\frac{T+1}{T+2}\approx 1$ and $\frac{1}{T}\approx 0$ and $P(1/2)=1/2$. Hence, the system moves with equal probability infinitesimally to the left or infinitesimally to the right.

My limited knowledge of stochastic dynamical systems prevents me from proving formally $$\alpha_t\xrightarrow{prob.}1/2$$ I would like to know if there is a way.

My attempts so far are:

a) Observe that

$$\alpha_T=\frac{1}{T+1}\alpha_0+\frac{\sum_{i=0}^{T-1}W_t}{T+1}$$

hence, for $T\to\infty$ the first term drops, while the second may be evaluated via a weak law of large numbers. The point is that, not only variables $W_t$ are all correlated, but also they are not identically distributed. Hence, I do not know any law of large numbers to apply here. In particular, while the covariance between the $W_t$ should go to zero, they do not have fixed mean and variance and I cannot apply the law for weakly stationary processes.

b) Try to study the evolution of $\alpha_t$ considering a Markov process on an extended state space, namely allowing the state to be $$(t,\alpha_t,w_t)\in\mathbb{N}\times[0,1]\times\{0,1\}$$ with transition specified by the following Markov kernel $$(t,\alpha_t,w_t)\in\mathbb{N}\times[0,1]\times\{0,1\}\mapsto\delta_{t+1}\otimes \{P(\alpha_t)\delta_{\frac{(t+1)\alpha_t+1}{t+2}}+(1-P(\alpha_t))\delta_{\frac{(t+1)\alpha_t}{t+2}}\}\otimes \{P(\alpha_t)\delta_{1}+(1-P(\alpha_t))\delta_{0}\}\in\Delta(\mathbb{N}\times[0,1]\times\{0,1\})$$ where $\otimes$ denotes the independent product of measures. I don't really know how to study such a Markov process, as my knowledge is limited to simpler cases. In particular, beside the complicated state space, the point is that starting from the same $\alpha$ at two different times $t,t'$ the support of the probability distribution describing the transitions from this point is different from time $t$ to time $t'$.

Any help would be more than welcome.

1

There are 1 best solutions below

0
On BEST ANSWER

Here is a proof sketch for an approach from first principles.


Fix $\varepsilon>0$ and say we want to show that $\Pr(\alpha_t<1/2 - \varepsilon)$ converges to 0 as $t\to\infty$.

Let's focus on your process $(\alpha_t)$ restricted to indices $t$ such that $\alpha_t\le 1/2-\varepsilon/2$. Intuitively, it is first going to look like a random walk with independent increments $\pm 1/t$ and bias at least $\delta=P(1/2-\varepsilon/2)$ towards the + direction. Since this random walk has expectation $\Omega(\log t)$ and standard deviation $O(1)$, by Chebyshev's inequality it is going to cross the $1/2-\varepsilon/2$ boundary eventually.

When it does, we just skip to the next time $t'$ such that $\alpha_{t'}\le 1/2-\varepsilon/2$. Crucially, the position at which the process jumps back will be $\alpha_{t'}\ge 1/2-\varepsilon/2-1/{t'}$. From this position, what is the probability that your process hits the boundary $1/2-\varepsilon$ before the boundary $1/2-\varepsilon/2$? The process is a submartingale, so by the optional stopping theorem this probability is at most $O_{\varepsilon}(1/{t'})$.


Putting everything together, fix some $t_0\ge 1/100\varepsilon$. By the first part of the argument, a.s. there exists $t_1\ge t_0$ such that $\alpha_{t_1}\ge 1/2-\varepsilon/2$. By the second part of the argument, for all $t_2\ge t_1$, $\Pr(\alpha_{t_2}<1/2-\varepsilon)\lesssim 1/t_1\le 1/t_0$. This implies that $\Pr(\alpha_t<1/2-\varepsilon)\lesssim 1/t_0$ for $t$ large enough, and taking $t_0\to \infty$ gives the result.