A scalar valued function is defined as $f(x)=x^TAx+b^Tx+c$ , where $A$ is a symmetric positive definite matrix with dimension $n\times n$ ; $b$ and $x$ are vectors of dimension $n\times 1$. Show that the minimum value of $f(x)$ will occur when $x$ equals to $-\frac{A^{-1}b}{2}$.
I found the answer of the same here. But I an unable to get the partial derivatives. Please any one explain the solution.
Related:
On
At the minimum the derivative will be zero. The derivative of $c$ will be zero, because it is a constant. The derivative of $x$ is $I_n$, so the derivative of $b^Tx$ is $b^T$ (numerator) resp $b$ (denominator).
The only tricky bit is the quadratic term.
If $y = Ax, \frac{\partial y}{\partial x} = A$ so, by the product rule, $\frac{\partial x^TAx}{\partial x} = \frac{\partial x^T}{\partial x}Ax+x^TA$
$\frac{\partial x^T}{\partial x}Ax=x^TA^T$ (see, e.g. your Wikipedia link), so collecting the terms in $x^T$, $\frac{\partial x^TAx}{\partial x} = x^T(A+A^T)=2x^TA$ (since $A$ is symmetric, $A=A^T$)
For it all to be equal to zero $2x^TA+b=0 \Rightarrow x^T=\frac{A^-1b}{2}$
Here is a solution with no calculus at all. It's the same process of completing the square that we learn in the first algebra course in high school with some linear algebra thrown in.
Let $q=\frac12 A^{-1}b$. Note that $$f(x) = (x+q)^\top A (x+q) + (c-q^\top Aq).$$ Notice that we use symmetry of $A$ to get $q^\top Ax= \frac12 b^\top (A^{-1}Ax) = \frac12 b^\top x$ and similarly for $x^\top Aq$. Since $A$ is positive definite, $y^\top Ay\ge 0$, with equality holding if and only if $y=0$. Thus, $f$ attains its minimum when $x+q=0$, i.e., when $x=-q=-\frac12 A^{-1}b$.