How to solve the quadratic form

Question

I am a physicist and I have a problem solving this \begin{equation} Q(x)=\frac{1}{2}(x,Ax)+(b,x)+c \end{equation} In a book it says that:

"The minimum of Q lies at $\bar{x}=-A^{-1}b$ and \begin{equation} Q(x)=Q(\bar{x})+\frac{1}{2}((x-\bar{x}),A(x-\bar{x})) \end{equation} How do I go to this? What how much is $Q(\bar{x})$?

Answer 1

You have $$ \frac{1}{2}((x-\bar{x}),A(x-\bar{x}))=\frac{1}{2}(x,Ax)-(x,A\bar{x})+\frac{1}{2}(\bar{x},A\bar{x}) $$ so that $$ \tag{1} \frac{1}{2}((x-\bar{x}),A(x-\bar{x}))+Q(\bar x)=\frac{1}{2}(x,Ax)-(x,A\bar{x})+(\bar{x},A\bar{x})+(b,\bar x)+c. $$ On the other hand: $$ (\bar x,A\bar x)+(b,\bar x)=(A^{-1}b,b)-(b,A^{-1}b)=0 \quad\hbox{and}\quad (x,A\bar{x})=-(x,b), $$ and plugging these into (1) yields the desired result.

Answer 2

The stated properties are analogues of those of the single-variable function $q:x\mapsto\frac12ax^2+bx+c$, for which the minimum occurs at $\bar x=-\frac ba$ (when $a\gt0$) and $q(x)=q(\bar x)+\frac12a(x-\bar x)^2$. It can be derived in a completely analogous fashion to the single-variable case.

Assuming that $A=A^T$, we have $\nabla Q(x)=\frac12(A+A^T)x+b=Ax+b$. The critical points occur where the gradient vanishes, so if $A$ is nonsingular this is at the unique point $\bar x=-A^{-1}b$. Since $Q(\bar x)$ is stated to be a minimum, we also have that $A$ is positive-definite: the graph of $Q$ is an elliptic paraboloid with vertex at $(\bar x; Q(\bar x))$.

The Hessian of $Q$ is $A$ and higher-order derivatives vanish, so by Taylor’s theorem, $$Q(x)=Q(\bar x)+\langle\nabla Q(\bar x),x-\bar x\rangle+\frac12\langle x-\bar x,A(x-\bar x)\rangle,$$ but $\nabla Q(\bar x)$ vanishes, leaving the desired equation.

As for the value of $Q(\bar x)$, finding that is a simple matter of substitution: $$\begin{align} Q(\bar x) &= \frac12\langle\bar x,A\bar x\rangle+\langle b,\bar x\rangle+c \\ &= \frac12\langle A^{-1}b,b\rangle-\langle b,A^{-1}b\rangle+c \\ &= c-\frac12\langle b,A^{-1}b\rangle. \end{align}$$ (Here we’ve again used the fact that $A=A^T$.) Compare this to the single-variable case, for which the vertex of the parabola is at $\left(-\frac ba,c-{b^2\over2a}\right)$.