Given positives $x$ and $y$ such that $2x + y \ne 2$ and $x + 2y \ne 2$, calculate the minimum value of $$\frac{(2x^2 + y)(4x + y^2)}{(2x + y - 2)^2} + \frac{(2y^2 + x)(4y + x^2)}{(2y + x - 2)^2} - 3(x + y)$$
We have that $$\frac{(2x^2 + y)(4x + y^2)}{(2x + y - 2)^2} + \frac{(2y^2 + x)(4y + x^2)}{(2y + x - 2)^2} - 3(x + y)$$
$$ = \sum_{\text{sym}}\left[\frac{(2x^2 + y)(4x + y^2)}{(2x + y - 2)^2} - (2x + y)\right] = 2 \cdot \sum_{\text{sym}}\frac{(xy - 2x - y)(xy - 4x - 2y + 2)}{(2x + y - 2)^2}$$
$$ = 2 \cdot \sum_{\text{sym}}\left[\left(\frac{xy - 2x - y}{2x + y - 2}\right)^2 - \frac{xy - 2x - y}{2x + y - 2}\right] \ge 2 \cdot \left[2 \cdot \left(-\frac{1}{4}\right)\right] = -1$$
The equal sign occurs when $\dfrac{xy - 2x - y}{2x + y - 2} = \dfrac{xy - 2y - x}{2y + x - 2} = \dfrac{1}{2}$
$$\iff \dfrac{xy - 2}{2x + y - 2} = \dfrac{xy - 2}{2y + x - 2} = \dfrac{3}{2}$$
$$\implies 2x + y - 2 = 2y + x - 2 \iff x = y$$
Furthermore, $\dfrac{xy - 2}{2x + y - 2} = \dfrac{xy - 2}{2y + x - 2} = \dfrac{3}{2}$
$$\iff 2(xy - 2) = 3(2x + y - 2) = 3(2y + x - 2)$$
$$\implies 2(x^2 - 2) - 3(3x - 2) = 2(y^2 - 2) - 3(3y - 2) = 0$$
$$\implies 2x^2 - 9x + 2 = 2y^2 - 9y + 2 = 0 \implies x = y = \frac{9 + \pm \sqrt{65}}{4}$$
I would like to know whether the above solution is correct and if not, whether there are any other solutions.
Thanks for your attention.
Solution by Nguyen Van Quy.
First, we have: If $a,b>0,\ a+b\neq 1$ then $$\dfrac{2(a^2+b)(a+b^2)}{(a+b-1)^2} \geq \dfrac{4a+4b-1}{2}.$$ Indeed, inequality equivalent to $$4(a^3+b^3+a^2b^2+ab)\geq (4a+4b-1)(a+b-1)^2,$$ or $$4\big[(a+b)^3-3ab(a+b)+a^2b^2+ab\big]\geq 4(a+b)^3-9(a+b)^2+6(a+b)-1,$$ or $$9(a+b)^2-6(a+b)(2ab+1)+(2ab+1)^2\geq 0,$$ or $$(3a+3b-2ab-1)^2\geq 0.$$ Done.
Now replace $(a,b)$ to $\left(x,\dfrac{y}{2}\right)$ and $\left(\dfrac{x}{2}, y\right),$ we get $$\dfrac{(2x^2+y)(4x+y^2)}{(2x+y-2)^2} \geq 2x+y-\dfrac{1}{2},$$ $$\dfrac{(2y^2+x)(4y+x^2)}{(x+2y-2)^2} \geq 2y+x-\frac{1}{2}.$$ Therefore $$\frac{(2x^2 + y)(4x + y^2)}{(2x + y - 2)^2} +\frac{(2y^2 + x)(4y + x^2)}{(2y + x - 2)^2} \geqslant 3(x + y) -1,$$ or $$\frac{(2x^2 + y)(4x + y^2)}{(2x + y - 2)^2} +\frac{(2y^2 + x)(4y + x^2)}{(2y + x - 2)^2} - 3(x + y) \geqslant -1.$$