Sum of the supremum and supremum of a sum

Question

Consider two real-valued functions of $\theta$, $f(\cdot): \Theta \subset\mathbb{R}\rightarrow \mathbb{R}$ and $g(\cdot):\Theta \subset \mathbb{R}\rightarrow \mathbb{R}$.

Is there any relation between

(1) $\sup_{\theta \in \Theta} (f(\theta)+g(\theta))$

and

(2) $\sup_{\theta \in \Theta} f(\theta)+\sup_{\theta \in \Theta} g(\theta)$

?

Could you provide some informal proof or intuition behind your answer?

Answer 1

See that $f(\theta) \le \sup_{\Theta}f(\theta)$ for all $\theta \in \Theta$ and also $g(\theta) \le \sup_{\Theta}g(\theta)$ for all $\theta \in \Theta$ and thus you'll have $f(\theta)+g(\theta) \le \sup_{\Theta}f(\theta)+\sup_{\Theta}g(\theta)$ for all $\theta \in \Theta$ and it follows that $\sup_{\Theta} \left ( f(\theta)+g(\theta) \right ) \le \sup_{\Theta}f(\theta)+\sup_{\Theta}g(\theta)$

Answer 2

To see that equality need not hold, consider $f(\theta) = 1 - g(\theta)$ with $g(\theta)$ positive monotone increasing. We clearly have that $$ \sup_\theta (f(\theta) + g(\theta)) = 1$$ while $$ \sup_\theta(f(\theta)) + \sup_\theta(g(\theta)) = 1 + \infty = \infty$$

Notice that the following inequality always hold $$\sup_\theta(f(\theta)+g(\theta)) \leq \sup_\theta(f(\theta)) + \sup_\theta(g(\theta)) $$ Intuitively, equality holds if and only if $f$ and $g$ achieve their maximal value at the same point $\theta$.

Answer 3

There is a relationship, and here's the intuition. It works best if we assume that $f$ and $g$ attain their supremums on $\Theta$, but you can extend the intuition to the general case. Let $\theta_f^*$ and $\theta_g^*$ be the values in $\Theta$ at which $f$ and $g$ achieve their supremums, respectively: $$ f(\theta_f^*) = \sup_{\theta \in \Theta} f(\theta), \qquad g(\theta_g^*) = \sup_{\theta \in \Theta} g(\theta). $$

If $f$ and $g$ attain their supremums at the same point $\theta_f^* = \theta_g^* =: \theta^*$, then to maximize $f(\theta) + g(\theta)$, we should pick $\theta = \theta^*$; we know this is right because picking any $\theta$ other than $\theta^*$ will decrease both $f$ and $g$, and hence $f+g$. So in this case, $$ \sup_{\theta \in \Theta} [f(\theta) + g(\theta)] = f(\theta^*) + g(\theta^*) = \sup_{\theta \in \Theta} f(\theta) + \sup_{\theta \in \Theta} g(\theta). $$

If $f$ and $g$ do not attain their supremums at the same point, that is, $\theta_f^* \neq \theta_g^*$, then again, the best possible value for $(f + g)(\theta)$ that we could hope for is $f(\theta_f^*) + g(\theta_g^*) = \sup_{\theta \in \Theta} f(\theta) + \sup_{\theta \in \Theta} g(\theta)$. But we can only evaluate $f+g$ at one particular $\theta$; maybe we should pick $\theta_f^*$ or maybe $\theta_g^*$ or some other $\theta$, but since $\theta_f^* \neq \theta_g^*$, we cannot simultaneously maximize $f$ and $g$ to maximize $f+g$. Hence, in this case, $$ \sup_{\theta \in \Theta} [f(\theta) + g(\theta)] \leq f(\theta_f^*) + f(\theta_g^*) = \sup_{\theta \in \Theta} f(\theta) + \sup_{\theta \in \Theta} g(\theta). $$ This is the inequality which holds most generally. @Siddharth Joshi shows how to prove it (without assuming $f$ or $g$ attains its supremum).

Answer 4

For $\alpha = \sup f \ge f(x)$ for all x, and $\beta = \sup g \ge f(x)$ for all x then $\alpha + \beta \ge f(x) + f(y)$ for all x. So $\sup f + \sup g = \alpha + \beta \ge \sup (f + g)$.

That's what others are calling "intuitive". I like to think of it as degree of limitation. We have more degree of freedom for f and g operating independently than together as a fixed sum (f + g). So the sum of the sups is greater (or equal) than the sup of the sums because we simply have more options.

But that's a pretty vague definition and if it isn't clear when I first say it, it'll just be confusing.

To show that equality might not hold, simply imagine f and g "get big" at different places. Imagine $f(x) < 1$ for all $x \ne 1$ but $f(1) = 1$ (example $f(x) = 1 - (x-1)^2$) $\sup f = 1$. Imagine $g(x) < 1$ for all $x \ne 0$ but $f(0) = 1$ (Ex. $g(x) = 1 - x^x$) $\sup g = 1$. But $f + g$ is always significantly less than 2. (In our examples $f + g = 1 - 2(x^2 - x) \le 3/2$)

Then $\sup f + \sup g > \sup (f + g)$. In our example $2 = \sup f + \sup g > \sup (f + g) = 3/2$

Or better yet let $g(x) = -f(x)$ where $f$ is bounded above and below but includes both positive and negative values. $f(x) + g(x) = 0$ so $\sup(f + g) = 0$ but $\sup f > 0$ and $\sup g = - \inf f > 0$.