Global minimum of $f(x)=\langle Ax,x \rangle +2 \langle x,b \rangle+c$

Question

Given that $A\in M_n(\Bbb R)$ is a symmetric positive-definite matrix, $b \in \Bbb R^n , c\in \Bbb R$ I need to find the global minimum of the function $$ f(x) =\langle Ax,x \rangle +2 \langle x,b \rangle + c.$$

To solve this I first wanted to solve $D_f(a)=0$ ($D_f(a)$ is the differential of $f$ at point $a$).

In order to find a local minimum, then show it is also global. But it seems like a lot of work if I define $A=(a)_{ij}$. Is there a shorter/better way?

Answer 1

First Method: Completing the square

Indeed, it using the fact that $A$ is symmetric, positive definite and invertible we have

\begin{align}0&\le \langle A(\color{red}{x+A^{-1}b}),\color{red}{x+A^{-1}b} \rangle \\&=\langle Ax,x \rangle+\langle Ax,A^{-1}b \rangle+ \langle b,x \rangle+\langle b,A^{-1}b \rangle \\&=\langle Ax,x \rangle+2\langle b,x \rangle+\langle b,A^{-1}b \rangle\\&=f(x) \color{blue}{-c+ \langle b,A^{-1}b\rangle} \end{align}

Thus for all $x\in \Bbb R^n$ we have, $$ f(x)\ge \color{blue}{c- \langle b,A^{-1}b\rangle} =\color{red}{f(-A^{-1}b)} $$ Whence, $$\color{blue}{\min_{x\in\Bbb R^n} f(x) =-\langle A^{-1}b,b \rangle+ c}$$

Second Method: By Calculus From here Differentiating $\langle Ax,x\rangle$ we have, $$Df(x)h = \langle Ax , h \rangle + \langle Ah , x \rangle+2b\cdot h\\= 2\langle Ax , h \rangle +2b\cdot h$$

since A is symmtric Thus, $$Df(x)= 2 Ax +2b $$

$$Df(x)= 0\Longleftrightarrow \color{red}{x= -A^{-1}b}$$

Moreover, since A is positive definite we get that the Hessian, $$D^2f(x)(h,h) = 2\langle Ah , h \rangle\ge 0$$

Hence,$ \color{red}{x= -A^{-1}b}$ is the only critical point of $f$ and the Hessian is positive definite. This means that, the global minimum of $f$ is attained at $ \color{red}{x= -A^{-1}b}$ hence,

$$\min f(x)=f( \color{red}{ -A^{-1}b}) =\langle AA^{-1}b,A^{-1}b \rangle -2 \langle A^{-1}b,b \rangle+c = -\langle A^{-1}b,b \rangle +c$$ That is $$\color{blue}{\min_{x\in\Bbb R^n} f(x) =-\langle A^{-1}b,b \rangle+ c}$$