derivative of square of dot product

2.4k Views Asked by At

I've got:

$\frac{\partial}{\partial \textbf{w}} \bigg( \textbf{x}^T\textbf{w}\bigg)^2$

when taking the derivative, does this result in

$2 (\underbrace{\textbf{x} ~\textbf{x}^T}_{\Sigma_x}) \textbf{w}^T$

or in the dot product of the $\textbf{x}$s

$2 (\underbrace{\textbf{x}^T ~\textbf{x}}_{scalar}) \textbf{w}^T$

2

There are 2 best solutions below

2
On BEST ANSWER

$$ \frac{\partial}{\partial \vec{w}}\left(\vec{x}^T \vec{w} \right)^2 = 2 (\vec{x}^T \vec{w})\frac{\partial}{\partial \vec{w}} (\vec{x}^T \vec{w}) =2 (\vec{x}^T \vec{w}) \vec{x}^T = 2 \vec{w}^T (\vec{x} \vec{x}^T) $$ So it is the second one you wrote down. If you are differentiating with respect to vectors, it is sometimes nicer to use indices, so in this case $x^i = \vec{x}$, $\vec{w}= w^i$, then: $$\frac{\partial}{\partial \vec{w}}\left(\vec{x}^T \vec{w} \right)^2 =\sum_{i} \frac{\partial}{\partial w^i}\left(\sum_j x^j w^j \right)^2 = \sum_i 2\left(\sum_j x^j w^j \right) \left(\sum_j x^j \underbrace{\frac{\partial w^j}{\partial w^i}}_{\delta_{ij}}\right) $$ $$ = \sum_i 2 \left(\sum_j x^j w^j \right) x^i = 2 \sum_i \sum_j x^j w^j x^i=2 \vec{w}^T (\vec{x} \vec{x}^T) $$ Which you can check is the same fairly easily.

0
On

If $\mathbf x$ and $\mathbf w$ are $n-$dimensional vectors, than

$(\mathbf x^T \mathbf w)^2=(x_1w_1+x_2w_2+\cdots +x_nw_n)^2=f^2$

is a scalar, and the derivative of a scalar by a vector is the vector: $$ \frac{\partial}{\partial \mathbf{w}}(f^2)=2f\frac{\partial f}{\partial \mathbf{w}}=\begin{pmatrix} 2(x_1w_1+x_2w_2+\cdots +x_nw_n)x_1\\ 2(x_1w_1+x_2w_2+\cdots +x_nw_n)x_2\\ \cdots\\ \cdots\\ 2(x_1w_1+x_2w_2+\cdots +x_nw_n)x_n \end{pmatrix}= $$ $$= 2(x_1w_1+x_2w_2+\cdots +x_nw_n) \begin{pmatrix}x_1\\x_2\\ \cdots\\ x_n \end{pmatrix}= 2(\mathbf {x}^T\mathbf {w})\mathbf {x} $$