Let $\{a,b\}\subset(0,1]$. Prove that: $$a^{b-a}+b^{a-b}+(a-b)^2\leq2$$
I don't see even how to begin the proof.
We can rewrite our inequality in the following form $$\frac{a^b}{a^a}+\frac{b^a}{b^b}+(a-b)^2\leq2$$ or $$(ab)^b+(ab)^a\leq(2-(a-b)^2)a^ab^b.$$ By Jensen easy to show that $a^ab^b\geq\left(\frac{a+b}{2}\right)^{a+b}$.
Thus, it remains to prove that $$\left(\frac{a+b}{2}\right)^{a+b}(2-(a-b)^2)\geq(ab)^b+(ab)^a$$ and I not sure that the last inequality is true.
Thank you!
Nice inequality. We need some lemmas first.
Let $\varphi\colon [0,1] \rightarrow \mathbb{R}$ be the function given by
$$\varphi(x) = \begin{cases} (x-1) \log \left(\frac{1-x}{2}\right)+(x+1) \log \left(\frac{x+1}{2}\right) & \text{if $x \neq 1$,} \\[1ex] 0 & \text{if $x = 1.$} \end{cases} $$
Lemma 1. $\varphi \geq 0$.
Proof. We have $$\lim\limits_{x \to 1} \varphi(x) = 0,$$ thus $\varphi$ is continuous and attains its minimum at a point $x \in [0,1]$. If $x \in \{0,1\}$, then $\varphi(x) = 0$ and we are done.
If $x \in (0,1)$, we have $$\varphi''(x)=\frac{2 x}{x^2-1} < 0.$$ Since $\varphi$ is differentiable at $x$ and $x$ is a minimum point, we also have $\varphi'(x) = 0$. By the second derivative test, $x$ is a strict local maximum point. This is inconsistent with $x$ being a global minimum point. Thus $x \in \{0,1\}$ and we are done. $$\tag*{$\Box$}$$
For all $x \in [0,1]$, we let $\psi_x\colon (0,1] \rightarrow \mathbb{R}$ be the function given by $$\psi_x(z) = (x-1) \log (z)+(x+1) \log (x+z).$$
Lemma 2. If $x \in [0,1]$, then $\psi_x \geq 0$.
Proof. Let $x \in [0,1]$.
If $x=1$, we have $\psi_x(z)=2 \log (z+1)\geq 0$ for all $z \in (0,1]$ and we are done.
If $x \in [0,1)$, then for all $z \in (0,1]$, we have $$\psi_x'(z) = \frac{2 x}{z (x+z)} \left(z-\frac{1-x}{2}\right).$$ For all $z \in (0,\frac{1-x}{2}]$, we have $\psi_x'(z) \leq 0$, thus $\psi_x$ is decreasing on $(0,\frac{1-x}{2}]$.
For all $z \in [\frac{1-x}{2},1]$, we have $\psi_x'(z) \geq 0$, thus $\psi_x$ is increasing on $[\frac{1-x}{2},1]$.
Since $\psi_x\left(\frac{1-x}{2}\right) = \varphi(x) \geq 0$ by Lemma 1, we are done. $$\tag*{$\Box$}$$
For all $x \in [0,1]$, we let $\gamma_x\colon (0,1] \rightarrow \mathbb{R}$ be the function given by $$\gamma_x(z) = x^2+z^x+(x+z)^{-x}.$$
Lemma 3. If $x \in [0,1]$, then $\gamma_x$ is increasing.
Proof. Let $x \in [0,1]$.
For all $z \in (0,1]$, we have \begin{align} \gamma_x'(z) = x \left(z^{x-1}-(x+z)^{-x-1}\right) &\geq 0 \\ \impliedby z^{x-1} &\geq (x+z)^{-x-1} \\ \iff (x-1) \log z &\geq (-x-1) \log(x+z) \\ \iff \psi_x(z) &\geq 0 \\ \impliedby &\text{Lemma 2}. \end{align} $$\tag*{$\Box$}$$
Claim. If $a,b \in (0,1]$, then $a^{b-a}+b^{a-b}+(a-b)^2 \leq 2$.
Proof. Let $a,b \in (0,1]$. Since the inequality is symmetric, we can assume $b \geq a$. Let $x = b-a \in [0,1)$.
We have \begin{align} a^{b-a}+b^{a-b}+(a-b)^2 &= a^{x}+(x+a)^{-x}+x^2 \\ &= \gamma_{x}(a) \\ \text{By Lemma 3:} \\ &\leq \gamma_{x}(1-x) \\[5pt] &= 1+x^2+(1-x)^x \\ \text{By Bernoulli's inequality:} \\ &\leq 2. \end{align} $$\tag*{$\Box$}$$