Matrix operations in optimization

38 Views Asked by At

I am taking an optimization course, but I am not very good with math.

In an example this function is given and gradient and hessian is asked. $$f(\vec{x})=\vec{x}^T\begin{bmatrix} 1 & 2 \\ 4 & 7 \end{bmatrix}\vec{x}+ \vec{x}^T\begin{bmatrix} 3 \\ 5 \end{bmatrix} + 6$$

In the solution, function is rewrited as this directly; $$f(\vec{x})=\frac{1}{2}\vec{x}^T\begin{bmatrix} 2 & 6 \\ 6 & 14 \end{bmatrix}\vec{x}+ \vec{x}^T\begin{bmatrix} 3 \\ 5 \end{bmatrix} + 6$$ Is this some kind of general form? What operations did they used to obtain second matrix?

I encountered way many of these $\vec{x}^TA\vec{x}$ formed expressions and many mathematical operations made with them.(derivate etc.) What are these expressions are called? I want to learn more about these things.

Thanks in advance. Sorry if my question is stupid or too basic.

1

There are 1 best solutions below

0
On BEST ANSWER

Quadratic models are typically expressed using the Hessian, which means it should have a symmetric matrix and there should be a factor $1/2$ in front of the quadratic term. To symmetrize $x^TAx$, replace $A$ with $(A+A')/2$ by noting that $x^TAx = (x^TAx + x^TA^Tx)/2 = \frac{1}{2}x^T(A+A^T)x$.