Differentiate vector function wrt vector

Question

I have a function $\frac{df(\mathbf{y})}{d\mathbf{y}}=\mathbf{y}g(\kappa)$ where $\kappa=||\mathbf{y}||_2$ and $g(\cdot)$ is a scalar function.

Thing is when I differentiate this function I get a scalar, whereas I am expecting a matrix: $$ \frac{d^2f(\mathbf{y})}{d\mathbf{y}d\mathbf{y}^T}=g(\kappa)+\mathbf{y}\frac{dg(\kappa)}{d\mathbf{y}}\\ =g(\kappa)+\mathbf{y}\frac{dg(\kappa)}{d\kappa}\frac{d\kappa}{d\mathbf{y}^T} $$

I'm pretty sure I'm using vector calculus wrong especially since the first term in the differential is a scalar. Wondering what am I doing wrong.

Answer 1

First, find the differential of $\kappa$ $$\eqalign{ \kappa^2 &= y^Ty \cr 2\,\kappa\,d\kappa &= 2\,y^Tdy \cr d\kappa &= \frac{y^Tdy}{\kappa} \cr }$$ and of $g(\kappa)$ $$\eqalign{ dg &= g'd\kappa \cr &= \frac{g'\,y^Tdy}{\kappa} \cr }$$ Then let $u=gy$ and find its differential $$\eqalign{ du &= g\,dy + y\,dg \cr &= g\,dy + y\Big(\frac{g'\,y^Tdy}{\kappa}\Big) \cr &= \Big(g\,I + \frac{g'\,yy^T}{\kappa}\Big)\,dy \cr }$$ Since $du=(\frac{\partial u}{\partial y^T})\,dy$, the derivative of $u$ is $$\eqalign{ \frac{\partial u}{\partial y^T} &= \,g\,I + \frac{g'}{\kappa}yy^T \cr }$$ which is the derivative you were asking about.

Note that both terms in the sum are square matrices.

Answer 2

Let $y_i$ represent the $i$-th coordinate of the $\mathbf{y}$ vector. Then,

$$ \frac{d^2f}{dy_j\,dy_i} = \frac{d}{dy_j}(\frac{df}{dy_i}) = \frac{d}{dy_j}\bigg(\hat{y}_i\,g(\kappa) + \mathbf{y}\,g'(\kappa)\,\frac{d\kappa}{dy_i}\bigg) $$

where $\hat{y}_i$ is the unit vector along the $i$-th direction, presumed independent of the coordinates of the vector $\mathbf{y}$. Performing the next derivative gives:

$$ \frac{d^2f}{dy_j\,dy_i} = \hat{y}_i\,g'(\kappa)\,\frac{d\kappa}{dy_j} + \hat{y}_j\,\,g'(\kappa)\,\frac{d\kappa}{dy_i} + \mathbf{y}\,g''(\kappa)\,\frac{d\kappa}{dy_j}\,\frac{d\kappa}{dy_i} + \mathbf{y}\,g'(\kappa)\,\frac{d^2\kappa}{dy_j\,dy_i} $$

or, more simply,

$$ \frac{d^2f}{dy_j\,dy_i} = \bigg(\hat{y}_i\frac{d\kappa}{dy_j} + \hat{y}_j\frac{d\kappa}{dy_i}\bigg)\,g'(\kappa) + \mathbf{y}\,\frac{d^2\kappa}{dy_j\,dy_i}\,g'(\kappa) + \mathbf{y}\,\frac{d\kappa}{dy_j}\,\frac{d\kappa}{dy_i}\,g''(\kappa) $$

And that's the $(ij)$-element of the matrix you were expecting. Note that the resulting matrix is symmetric.