what is the Jacobian of this function?

Question

$f(x)=xx^{T}$ for $x\in{\Bbb{R}^{n}}$

my solutions

the rest of my solution

Answer 1

I realize you are asking about differentiation of matrix with respect to a vector.

The answer I had given was classical differentiation (see below). I keep it because I think you have first to understand this notion before considering the more complex differentiation with respect to a vector.

The easiest way to apprehend a jacobian is to consider its action on an "infinitesimal element".

Let $H$ be a vector (think to it as a small increment).

Write :

$$f(X+H)=(X+H)(X+H)^T=(X+H)(X^T+H^T)$$

$$f(X+H)=\underbrace{XX^T}_{f(X)}+\underbrace{XH^T+HX^T}_{J_X(f)(H)}+HH^T$$

in which we have recognized a Taylor expansion.

For example, in the 2D case, with notations $X=\binom{x}{y}$ and $H=\binom{h}{k}$:

$$J_X(f)(H)=XH^T+HX^T=\binom{x}{y}(h \ \ k)+\binom{h}{k}(x \ \ y)=\begin{pmatrix}2xh&xk+yh\\xk+yh&2yk\end{pmatrix}\tag{1}$$

Said with a sentence: the jacobian $J_X$ of $f$ in point $X$ acts on differential element $H$. Do not isolate it from this differential element.

Remark: (1) is an exact Taylor expansion with second order term:

$$HH^T=\begin{pmatrix}h^2&hk\\ hk&k^2\end{pmatrix}$$ (with second order entries, which is quite normal).

Answer 2

Let $\mathbf{A}=\mathbf{x}\mathbf{x}^T$ be a $N\times N$ matrix. The $(i,j)$-th element is $$ A_{ij} = x_i x_j = \mathbf{e}_i^T \mathbf{x}: \mathbf{e}_j^T \mathbf{x} $$ Here the colon operator denotes the Frobenius inner product.

The differential gives $$ dA_{ij} = \mathbf{E}_{ij}\mathbf{x}: d\mathbf{x} $$ with the symmetric matrix $\mathbf{E}_{ij}= \mathbf{e}_i \mathbf{e}_j^T+ \mathbf{e}_j \mathbf{e}_i^T$.

We deduce the derivative term $$ \frac{\partial A_{ij}}{\partial x_k} = \mathbf{e}_k^T \mathbf{E}_{ij}\mathbf{x} $$ which is indeed a $N\times N \times N$ third-order tensor as explained by Jean-Marie.