being $\mathbf{w}$ a vector, how do I calculate the derivative of $\mathbf{w}^T\mathbf{w}$?

Question

Let's say that I have a vector $\mathbf{w}$.

How can I calculate the derivative in the following expression?

$\frac{\mathrm{d}}{\mathrm{d}\mathbf{w}}\mathbf{w}^T\mathbf{w}$

Update: found these useful definitions

Answer 1

It is easier to see it in component form.

Let $\hat{x_i}$ be the unit vector in the $i$-th direction, then we can express a vector as

$$\mathbf{w}=\sum_{i=1}^{n}w_i \hat{x_i} \tag{1}$$

And

$$\frac{d}{d\mathbf{w}}=\sum_{i=1}^{n}\hat{x}^T_i \frac{d}{dw_i} \tag{2}$$

So

$$\mathbf{w}^T \mathbf{w}=\sum_{i=1}^{n}w_i^2 \tag{3}$$

$$\frac{d}{d\mathbf{w}}(\mathbf{w}^T \mathbf{w})=\sum_{i=1}^{n}2w_i\hat{x}^T_i =2\mathbf{w}^T\tag{4}$$

EDIT: I made a minor correction ($\hat{x}_i$ to $\hat{x}^T_i$)in (2) and (4) based on rych's suggestion. Now the final results is $2\mathbf{w}^T$

Answer 2

In order to find the derivative let us calculate the h-linear term, $$\begin{equation} \begin{split} (w+h)^T(w+h)-w^Tw&=(w^T+h^T)(w+h)-w^Tw\approx\\ &\approx h^Tw+w^Th= (h^Tw)^T+w^Th=2w^Th \end{split} \end{equation}$$ Hence the derivative is: $2w^T$. Rather than $2w$. Indeed the gradient is not really a column- but a row-vector (or covector, or dual vector).