I would like to know if I have a good understanding of what explains the following result
Let $V$ be an Hilbert space, $J$ a strongly convex and differentiable functional. Assume $J’$ is locally lipschitz. We define $u_{n+1} = u_n + \mu_{n}J’(u_n)$ where $\mu_n$ is chosen such that $J(u_{n+1}) = \inf_{\mu\in\mathbb{R}}J(u_n + \mu J’(u_n))=\inf_{\mu\in\mathbb{R}}f(\mu)$. Then $J(u_n)$ converges to $J(u)$ the minimum of $J$ over $V$.
Proof : First we remark that the minimization problem has a solution since $J$ is convex and differentiable on a closed and convex set. Now we notice that by definition of $f(x)$ we have that this function is differentiable with derivative given by $J’(u_n + xJ’(u_n))(J’(u_n))$ and convex. Thus $\inf_{\mu\in\mathbb{R}}f(\mu)$ is reached and by the first order condition gives, at that point, $f’(\mu)= J’(u_n + \mu J’(u_n))(J’(u_n)) = J’(u_{n+1})(J’(u_n))= \langle J’(u_{n+1}), J’(u_n)\rangle = 0$. Where we use the Riesz representation theorem to identify the derivative with its corresponding vector in $V$ that we denote $J’(.)$.
By the strong convexity, we know there exists some $\alpha>0$ such that for all $n$ we have
$$ J(u_n)\geq J(u_{n+1}) + \langle J’(u_{n+1}), u_{n}-u_{n+1}\rangle + \frac{\alpha}{2}\lVert u_{n}-u_{n+1}\rVert^{2} $$
But $u_{n}-u_{n+1} = \mu_{n}J’(u_n)$ so $langle J’(u_{n+1}), u_{n}-u_{n+1}\rangle=0$. Thus we have that the sequence $(J(u_n))_n$ is decreasing but it is bounded below by $\inf_{u\in V}J(v)$ so it converges.
Now from the convergence (the cauchyness more precisely) of $(J(u_n))_n$ and again by strong convexity we have the existence of an $\epsilon>0$ such that for all $n,m\geq N$ we have
$$ \epsilon\geq \lvert J(u_n) - J(u_m)\rvert\geq\frac{\alpha}{2}\lVert u_n - u_m\rVert^{2} $$
First this shows that $(u_n)_n$ is Cauchy and thus converges. Second, it implies that for all $n\geq N$ we have
$$ \lVert u_n \rVert\leq \sqrt{\frac{2\epsilon}{\alpha}} + \lVert u_N \rVert $$
For the term before the rank $N$ take the maximum over these terms and we get that the sequence is bounded by say $M$ . Now we use the fact that $J’$ is locally lipschitz : take $M$, $\exists C$ such that
$$ \lVert x\rVert + \lVert y\rVert\geq M\implies \lVert J(x)-J(y)\rVert\leq C\lVert x - y \rVert $$
Then we get using the orthogonality relation $\langle J’(u_{n+1}), J’(u_n)\rangle=0$ that
$$ \lVert J’(u_n)\rVert^{2} \leq \lVert J’(u_n)\rVert^{2} + \lVert J’(u_{n+1})\rVert^{2} = \lVert J’(u_n) - J’(u_{n+1})\rVert^{2}\leq C\lVert u_n - u_{n+1}\rVert^{2} $$
Which shows, using the convergence of $u_n$ that $(J’(u_n))$ converges to the null vector.
Now using gain the fact that $J$ is strongly convex and the fact that at $\bar{u}$ the minimum oh $J$ we have $\langle J’(\bar{u}), h\rangle = 0$ for all $h\in V$ we get
$$ 0\leq\alpha \lVert u_n -\bar{u}\rVert^{2}\leq \langle J’(u_n) - J’(\bar{u}), u_n - \bar{u}\rangle\leq\lVert J’(u_n)\rVert \lVert u_n - \bar{u}\rVert $$
Where the last inequality follows from Cauchy Schwarz. Dividing by $\lVert u_n - \bar{u}\rVert$ and taking the limit we have our conclusion.
I would like to know if this is correct and if all arguments used are clear.
Thank you !
Edit
It seems that there is something not clear here
$$ \epsilon\geq \lvert J(u_n) - J(u_m)\rvert\geq\frac{\alpha}{2}\lVert u_n - u_m\rVert^{2} $$
Indeed the term $J’(u_m)(J’(u_n))$ has disappeared. At first my argument was this one $J’(u_{n+1})(J’(u_n))= \langle J’(u_{n+1}), J’(u_n)\rangle = 0$ but when I try to show this holds for $m>n$ inductively I cannot.