Restricted Strong Smoothness vs Strong Smoothness

28 Views Asked by At

This paper defines the Restricted Strong Smoothness as follows:

A differentiable function $f: \mathbb{R}^n \to \mathbb{R}$ is said to be Restricted Strongly Smooth (RSS) with modulus $L_s>0$ or is $L_s$-RSS if

$$ f(\mathbf{y}) \leq f(\mathbf{x}) + \langle \nabla f(\mathbf{x}) , \mathbf{y}-\mathbf{x} \rangle + \frac{L_{s}}{2}\|\mathbf{y}-\mathbf{x}\|_2^2 $$

holds for all $\mathbf{x},\mathbf{y} \in \mathbb{R}^n$ such that $\|\mathbf{x}\|_0 \leq s,\|\mathbf{y}\|_0\leq s$ where $1 \leq s<n$ is the sparsity level and $\|\cdot\|_0$ is the zero-norm defined as the count of nonzero elements in a vector. However, Strong Smoothness (global one) is defined as follows: A differentiable function $f: \mathbb{R}^n \to \mathbb{R}$ is said to be Strongly Smooth (SS) with modulus $L>0$ or is $L$-SS if

$$ f(\mathbf{y}) \leq f(\mathbf{x}) + \langle \nabla f(\mathbf{x}) , \mathbf{y}-\mathbf{x} \rangle + \frac{L}{2}\|\mathbf{y}-\mathbf{x}\|_2^2 $$

holds for all $\mathbf{x},\mathbf{y} \in \mathbb{R}^n$.

Question

Is there a function that is RSS for some $s$ but it is not Strongly Smooth? For example, is there a function defined on $\mathbb{R}^2$ that is RSS with $s=1$ but does not have a second order bound? Or, maybe in $\mathbb{R}^3$, where $f$ is an RSS function with $s=1$ or $s=2$?

1

There are 1 best solutions below

2
On

Consider a function that is zero on all points $x$ for which $\|x\|_0\neq n$, and for which the derivative vanishes at such points as well. Such a function is clearly what you call $L_{s}$-RSS for any $s<n$ and any $L$.

However outside of the axes $\|x\|_0<n$ you are more or less free to let your function grow however you like, in particular if you make it grow faster than a quadratic function there is no hope that you can find an $L$ for which it satisfies the condition you have called $L$-SS.

Example: $$f:\Bbb R^n\to\Bbb R, \qquad (x_1,...,x_n)\mapsto x_1^2\cdots x_n^2,$$ i.e. $f(x)$ is the product of the squares of the components of $x$.