I am currently taking Optimization Theory and the book I'm using has almost no examples and there aren't any answers to the exercise questions. It's been a while since I worked with matrices and am even having difficulty searching for similar problems.
The question is as follows: Calculate ${f_x(x)}$ and ${f_{xx}(x)}$ for the following ${f(x)}$:
i.) ${f(x) = ({a^T}x)({b^T}x),}$ with ${ a,b\in {\mathbb{R}^n}}$
ii.) ${f(x)} = {x^T}Ax,$ with $A \in {\mathbb{S}^n}$. What happens if $A \in {\mathbb{R}^{n \times n}}$ is not symmetric?
iii.) ${f(x)} = {{\| Ax-b \|}^2},$ with $A \in {\mathbb{R}^{m\times n}}$ and $b \in {\mathbb{R}^m}$
iv.) ${f(x)}={sin({a^T}x)},$ with $a \in {\mathbb{R}^n}$
Unfortunately, I am having a hard time remembering how to work with stuff like this, so any help is greatly appreciated. I did find this: Differentiation of the euclidean norm of $\Vert Ax+b\Vert^{2}$ but still don't rally know what's going on.
My work so far is as follows:
By simply following the power rule, I got
i.) ${f_{x}(x)}= 2{a^T}{b^T}x$
${f_{xx}(x)}= 2{a^T}{b^T}$
And for ii.) ${f_x(x)}=d({x^T})Ax + {x^T}d(x) = {(dx)^T}Ax + {x^T}A(dx) = {(dx)^T}Ax + {(dx)^T}{A^T}x = {(dx)^T}(A+{A^T}x)$
But again, I am not confident in what I am doing. And, now I'm stuck on (iii.). Any help/suggestions would be greatly appreciated.