I have the following function
$$f(x) = \frac 14 (x^T A x)^2 + b^T x$$
where $A$ is symmetric and positive definite, and I proved its convexity. However, when I try to minimize it, I get stuck. After I take its gradient w.r.t. vector $x$ and equate it to $0$
$$A x x^T A x = -b$$
I can't find a way to move both $A$'s to the right hand side of the equation. What trick should I use to move them? The rest is straightforward: just a bunch of norm and normalized vector stuff. Thanks.
The correct optimality condition is $$(x^TAx)Ax + b = 0$$ Since $A$ is positive definite, $(x^TAx)Ax$ will be zero if and only if $x=0$. Therefore, the only way $x=0$ can be the solution is if $b=0$ as well. On the other hand, if $b\neq 0$, then $x\neq 0$, and $$x = -(x^TAx)^{-1} A^{-1} b$$ At first this may not seem helpful, but it helps reveal an important point: since $x^TAx$ is a scalar, $$x = \lambda A^{-1} b$$ for some value of $\lambda$. (Indeed, we didn't need to assume $x\neq 0$ to see this.) Let's plug that into the original optimality conditions; we get $$\lambda^3(b^TA^{-1}b)b + b = \left(\lambda^3(b^TA^{-1}b) + 1\right) b = 0 \quad\Longrightarrow\quad \lambda = -(b^TA^{-1}b)^{-1/3}$$ Therefore, $$x = -(b^TA^{-1}b)^{-1/3} A^{-1} b$$