Help needed to understand the formulation of the robbinson monro theorem.

76 Views Asked by At

I have troubles understanding the Robbins Monro theorem form stochastic approximation. I came across the theorem some time ago, but unfortunately few doubts still remain. First of all, let me begin by stating the formulation of the theorem. Taken from the book of T. M. Wasan "Stochastic Approximation,Cambridge University Press".

enter image description here

What I do not understand is the exact meaning of the conditional probabilities. Or more precisely, what is the exact meaning of $Y_n$ from

$$ P[Y_n=1|X_1,X_2,....,X_n,Y_1,....,Y_{n-1}]=M(X_n), $$ $$ P[Y_n=0|X_1,X_2,....,X_n,Y_1,....,Y_{n-1}]=1-M(X_n). $$

I would appreciate your help.

1

There are 1 best solutions below

2
On BEST ANSWER

A more down-to-earth description of the procedure is the following. $X_1=x_1$ is given (and presumably non-random). Now toss a coin whose Heads probability is $M(X_1)$, and record the result as $Y_1=1$ if Heads are thrown and $Y_1=0$ if tails are thrown. Then define $X_2$ via the formula $X_2=X_1-(Y_1-\alpha)$. Continue recursively: having obtained $X_1,X_2,\ldots,X_n$ and $Y_1,Y_2,\ldots Y_{n-1}$, generate $Y_n$ by tossing a coin with Heads probability $M(X_n)$, and then define $X_{n+1}$ to be $X_n-(Y_n-\alpha)/n$, etc.

Do note that $M$ is to be a distribution function, hence non-decreasing, right-continuous, and with values in the interval $[0,1]$. Thus $M$ cannot be the exponential function $\theta\mapsto e^\theta$, but it might be $\theta\mapsto 1-e^{-\theta}$, for example.