If $f$ and $g$ are functions with the same stationary points satisfying the PL Inequality does it hold for f + g?

112 Views Asked by At

The Polyak-Łojasiewicz (PL) inequality is an important condition in optimization, known for its role in analyzing convergence rates to global minima. It's given for a function $h$ as:

$$ \frac{1}{2} \|\nabla h(x)\|^2 \geq \mu (h(x) - h(x^*)) $$

where $ \mu > 0 $ is a constant, $\nabla h(x) $ is the gradient at $ x $, and $x^*$ is a global minimizer of $ h $.

Consider two functions, $f$ and $ g$, both satisfying the PL inequality, with the same stationary points $ \mathcal{X}^*_f = \mathcal{X}^*_g = \mathcal{X}^*$. Thus, if $ x^* \in \mathcal{X}^*$ minimizes $ f$, it also minimizes $g$.

Without the condition that the stationary points are shared it is easy to come up with functions such that $f + g$ does not satisfy the PL inequality, see for example this question which gives counterexamples showing invexity is not closed under addition.

The question is whether, under these additional assumptions, $ f + g $ satisfies the PL inequality? Intuitively you might think it does.

The obvious way to start here is to say:

$$ \frac{1}{2} (\|\nabla f(x)\|^2 + \|\nabla g(x)\|^2 ) \geq \mu_{f + g} (f(x) - f(x^*) + g(x) - g(x^*))$$

But since:

$$ \|\nabla f(x)\|^2 + \|\nabla g(x)\|^2 \geq \|\nabla f(x) + \nabla g(x)\|^2 $$

This does not immediately imply the desired inequality for $f + g$

$$ \frac{1}{2} \|\nabla f(x) + \nabla g(x)\|^2 \geq \mu_{f + g} (f(x) - f(x^*) + g(x) - g(x^*)) $$

This makes me feel like there should be a counterexample, probably something with a saddle point, but I have not been able to construct one.

If this does hold I would be very interested in whether we can relax the assumption so that instead of $\mathcal{X}^*_f = \mathcal{X}^*_g$ we have $\mathcal{X}^*_f \cap \mathcal{X}^*_g \neq \emptyset$.

I would also be interested in knowing whether changing to the Łojasiewicz inequality or invexity would change the conclusion here. I think not, but the subtleties between the classes of functions satisfying these three inequalities are a bit lost on me.