Scaled gradient descent

837 Views Asked by At

Consider the unconstrained minimization \begin{align*} \min_{x\in\mathbb{R}^n}f(x) \end{align*} One iterative approach to obtaining a solution is to use the gradient descent algorithm. This algorithm generates iterates via the following rule (assuming that $f$ is differentiable) \begin{align*} x_{k+1} = x_k - \alpha_k\nabla f(x_k) \end{align*} Now consider a different algorithm, termed the scaled gradient descent algorithm. This algorithm takes the form \begin{align*} x_{k+1} = x_k - \alpha_kD_k^{-1}\nabla f(x_k) \end{align*} where $D_k$ is a positive-definite matrix.

Question: Does anyone know of any more literature on the scaled gradient descent algorithm, particularly its convergence? I cannot seem to find much online -- perhaps I am using an incorrect keyword?