The (implicit) Euler scheme for gradient descent can be defined for a functional $f: \mathbb{R}^n \rightarrow \mathbb{R}$ by the sequence $\{x^k\}_{k\geq 0}$ where $x^0$ is given and $$ x^{k+1} = x^k - \eta \nabla f(x^k), \;\;\; k \geq 1.$$ Here $\eta$ is the step size. We know (under some conditions):
As $\eta \rightarrow 0$, the scheme converges to the gradient flow $x'(t) = -\nabla f(x(t))$ (in the sense that the interpolation of the $\{x^k\}_{k\geq 0}$ converges to $x(t)$ in some function space).
If $f$ is convex, then this is equivalent to the proximal gradient descent step $$ x^{k+1} = \arg \min_x \{ \frac12 ||x - x^k||^2 + \eta f(x)\}$$
(this is because, if $f$ is convex, thus $ \frac12 ||x - x^k||^2 + \eta f(x)$ is convex and has a unique argmin. Setting the derivative to 0, we get that the argmin is $x^k - \eta \nabla f(x^k)$.)
Now I am wondering about the coordinate-wise Euler scheme: let the sequence $\{x^k\}_{k\geq 0}$ where $x^k = (x^k_1, \dots, x^k_n)$ be defined as $\{\{x^k_i\}_{i=1}^n\}_{k \geq 1}$ where $$ x^{k+1}_i = x^k_i - \eta \frac{\partial }{\partial x_i} f(x^k)$$
I have two questions:
- in the small step size limit $\eta \rightarrow 0$, does the above coordinate-wise scheme converge to the gradient flow $x'(t) = - \nabla f(x(t))$?
- is this equivalent to the analogous proximal step $$ x^{k+1}_i = \arg \min_{x_i} \{ \frac12 ||x_i - x^k_i||^2 + \eta f(x^k_1, \dots, x_i, \dots, x^k_n)\}?$$