Definition of $L$-smooth function $f$ after scaling with a positive scalar $\alpha$, i.e., $\alpha f$

119 Views Asked by At

By definition the gradient of a function $f$ is $L$-Lipschitz continuous if:

\begin{align} \tag 1 \|\nabla f(x) - \nabla f(y)\| \leq L\|x- y\|. \end{align}

For the scaled function $g:=\alpha f$, for $\alpha > 0$, \begin{align} \tag 2 \left\| \nabla g(x) - \nabla g(y) \right\| &\leq L^\prime \|x- y\| \\ \tag 3 \left\| \left[\alpha \nabla f(x) - \alpha \nabla f(y) \right]\right\| &\leq L^\prime \|x- y\|\\ \tag 4 \left\| \alpha \left[ \nabla f(x) - \nabla f(y) \right]\right\| &\leq L^\prime \|x- y\|. \end{align}

Question: What is the relation between $L^\prime$ and $L$?


Update:

Following Arctic Char's comment, it is straight forward to show that \begin{align} \left\| \alpha \left[ \nabla f(x) - \nabla f(y) \right]\right\| &\leq L^\prime \|x- y\| \\ \alpha \left\| \left[ \nabla f(x) - \nabla f(y) \right]\right\| &\leq L^\prime \|x- y\| \\ \left\| \left[ \nabla f(x) - \nabla f(y) \right]\right\| &\leq \frac{L^\prime}{\alpha } \|x- y\|. \end{align}

Thus, $L^\prime = \alpha L $.