Derivative with respect to vector

51 Views Asked by At

I want to calculate the following derivative

$\frac{\partial(\textbf{x}^2)^T\textbf{b}-\textbf{c}^T\textbf{x}}{\partial \textbf{x}}, \quad \textbf{x},\textbf{b},\textbf{c}\in\mathbb{R}^{n}$ (column vectors)

it must be something like the following

$-\textbf{c}+\frac{\partial(\textbf{x}^2)^T\textbf{b}}{\partial \textbf{x}}.$

If I calculate this for a specific example i get this element wise solution

$\begin{bmatrix} -c_1\\ \vdots\\ -c_n \end{bmatrix} + \begin{bmatrix} 2 x_1 b_1\\ \vdots\\ 2 x_n b_n \end{bmatrix}$.

Is there a way to write the above as a matrix/vector operation, specifically the last term?

1

There are 1 best solutions below

4
On BEST ANSWER

You can do it by components

$$ f({\bf x}) = \sum_i x_i^2b_i - c_i x_i $$

So that

$$ \frac{\partial f}{\partial x_j} = \sum_i2x_i\delta_{ij}b_i -c_i\delta_{ij} = 2x_jb_j - c_j \equiv \left(\frac{\partial f}{\partial {\bf x}}\right)_j $$

In matrix form

$$ \frac{\partial f}{\partial {\bf x}} = \left(\begin{array}{c} 2x_1b_1 - c_1 \\ 2x_2b_2 - c_2 \\ \vdots \\ 2x_nb_n - c_n \end{array}\right) $$

You can then express this as

$$ \frac{\partial f}{\partial {\bf x}} = 2 \left(\begin{array}{cccc} b_1 & & & \\ & b_2 & & \\ & & \ddots & \\ & & & b_n \end{array}\right) \left(\begin{array}{c} x_1 \\ x_2 \\ \vdots \\ x_n \end{array}\right) - \left(\begin{array}{c} c_1 \\ c_2 \\ \vdots \\ c_n \end{array}\right) = 2{\rm diag}({\bf b})^T {\bf x} - {\bf c} $$