Let: $a,b,c>0$. Find the maximum value of: $$P= \frac{2ab+3b^2}{(a+3b)^2}+\frac{2bc+3c^2}{(b+3c)^2}+\frac{2ca+a^2}{(c+3a)^2}$$
Here are my try:
I tried to use tangent line trick, then I got:
$$\frac{2ab+3b^2}{(a+3b)^2} \leq \frac{1}{4}(a-b)+\frac{5}{16}$$
which is only true if $a \geq b$
I tried to use Cauchy - Schwarz but it seem impossible.
Let $\dfrac{a}{b}, \dfrac bc, \dfrac ca = x,y,z$ respectively, so you have $xyz=1, x,y, z >0.$ Then, prove: $$\dfrac{2x+3}{(x+3)^2}\leq\dfrac{11-x}{32}$$ and after summing up, you just need to show: $$x+y+z\geq 3$$ which is just AM-GM.
The comment below is right - this is only good for $x,y,z\leq 3.$ But if one of them, say $x$, is larger than $3,$ then: $$\dfrac{2x+3}{(x+3)^2}<\dfrac{9}{36}$$ and so: $$\sum_{x,y,z} \dfrac{2x+3}{(x+3)^2}< \dfrac{9}{36} + \dfrac 13 + \dfrac 13 = \dfrac{11}{12}<\dfrac{15}{16},$$ since $$\dfrac{2x+3}{(x+3)^2} = \dfrac{2x+3}{x^2+3(2x+3)}<\dfrac{1}{3}.$$