Let $a$ and $b$ be real numbers. Let $r,$ $s,$ and $t$ be the roots of $f(x) = x^3 + ax^2 + bx - 1,$ and then let $g(x) = x^3 + mx^2 + nx + p$ be a polynomial with roots $r^2,$ $s^2,$ and $t^2.$ If $g(-1) = -5,$ find the greatest possible value for $b.$
I got $rst=1$ and $(r^2+1)(s^2+1)(t^2+1)=5$ and you have to maximize $rs+st+tr$.
An alternative to using the formulae for the roots is to let $y=x^2$.
Then $(y+b)x+ay-1=0$ and so $(y+b)^2y=(1-ay)^2$. Then $$g(y)=y^3+(2b-a^2)y^2+(b^2+2a)y-1.$$
substituting $g(-1)=-5$ gives $$-5=-1+(2b-a^2)-(b^2+2a)-1$$
Then $(a+1)^2+(b-1)^2=5$ and $b_{\text {max}}=1+\sqrt5$.