So I have been given the following problem for homework, and have no idea where I should start on it. I have the function $f(u)$ given as follows:
$f(u) = u^{T}Au - 2ub$
Where $u$ is $nx1$, $u^T$ is the vector transpose of $u$ A is $nxn$ and SPD, and $b$ is $nx1$. The problem, then, is to find the gradient of $f$ with respect to $u$. I've taken Vector Calculus and Linear Algebra and still have no idea where to start on this. Any help you could give would be great.
First of all, $f(u)$ is a scalar function of the vector $u$. So, $\nabla_uf(u)$ will be a vector valued function of $u.$
There are several ways of proceeding, I'll present the way using components. So, we first write $f(u)$ using components as follows. $$f(u)=\sum\limits_{k=1}^{n}\sum\limits_{l=1}^{n}\left(u_kA_{kl}u_l-2u_kb_k\right)$$ Then the $i$-th component of the vector $\nabla_uf(u)$ is given by the following. $$\left(\nabla_uf(u)\right)_i=\frac{\partial f}{\partial u_i}$$ So by using the component form for $f$, we can start to calculate. $$\left(\nabla_uf(u)\right)_i=\frac{\partial}{\partial u_i}\left(\sum\limits_{k=1}^{n}\sum\limits_{l=1}^{n}\left(u_kA_{kl}u_l-2u_kb_k\right)\right)$$ That's the background to get you started.
A couple of useful things to remember.
$\qquad$ 1) $\frac{\partial u_k}{\partial u_i}=\delta_{ki}$. Here $\delta_{ki}$ is the Kronecker delta.
$\qquad$ 2) The matrix $A$ is symmetric. So $A_{ki}=A_{ik}$.