Derivates of a vector in respect to the elements

Question

Find the derivative of

$(a) f(x)= \frac{1}{x_3} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}$

$(b) f(x)=trace(xx^T)$

where $ x_1 , x_2, x_3 $ are the first three elements of x.

Answer 1

(a) The gradient of a vector valued function is a matrix. For $$f(x_1,x_2,x_3)=\begin{bmatrix}f_1(x_1,x_2,x_3)\\f_2(x_1,x_2,x_3)\end{bmatrix}$$ it is $$Df=\begin{bmatrix}\frac{\partial f_1}{\partial x_1}&\frac{\partial f_1}{\partial x_2}&\frac{\partial f_1}{\partial x_3}\\\frac{\partial f_2}{\partial x_1}&\frac{\partial f_2}{\partial x_2}&\frac{\partial f_2}{\partial x_3}\end{bmatrix}$$

So in your case

$$Df = \begin{bmatrix}\frac{1}{x_3}&0&-\frac{x_1}{x_3^2}\\0&\frac{1}{x^3}&-\frac{x_2}{x_3^2}\end{bmatrix}$$

(b) the function $g(x)=trace(xx^T)$ can be written as

$$g(x)=trace(xx^T)=\sum_{i=1}^n x_i^2\in\mathbb{R}$$

i assume here that $x\in\mathbb{R}^n$

The derivative of a real-valued expression is a vector containing all the partial derivatives, the 'gradient' of the function $g$. That means

$$Dg(x)= \begin{bmatrix}\frac{\partial g}{\partial x_1}\\...\\\frac{\partial g}{\partial x_n}\end{bmatrix}$$

The gradient is also written as $\nabla g(x)$. In your case

$$\nabla g(x)=Dg(x)= \begin{bmatrix}2x_1\\2x_2\\...\\2x_n\end{bmatrix}$$