What is the derivative of $(\mathbf{I} - \mathbf{x x}^T) \,\mathbf{x}$ with respect to $\mathbf{x}$?

Question

What is the derivative of $(\mathbf{I} - \mathbf{x} \mathbf{x}^T ) \, \mathbf{x}$ with respect to $\mathbf{x}\in \mathbb{R}^n$?

I think the result should in $\mathbb{R}^{n\times n}$, but I failed when calculate the first part.

The first part $\mathbf{I} - \mathbf{xx}^{T}$ is a $n\times n$ matrix and the derivative belongs to $\mathbb{R}^{n^2\times n}$.

So, is there anything wrong with the process?

Answer 1

The mapping $f(x):=(I-xx^T)x$ is a mapping from $\mathbb R^n$ to $\mathbb R^n$. It can be simplified to $$ f(x) = (I-xx^T)x = x - |x|_2^2 x. $$ Its directional derivative at $x$ in direction $h$ is given by $$ f'(x;h) = h - |x|_2^2 h - 2x^Th\cdot x\\ =h - |x|_2^2 h - 2xx^Th\\ =(I-|x|_2^2 I -2xx^T)h. $$ hence the derivative is given by the matrix $$ f'(x) = I-2xx^T- |x|_2^2 I, $$

Answer 2

Define the variables $$\eqalign{ A &= (I-xx^T) &\implies dA = -(dx\,x^T+x\,dx^T) \cr b &= (x+a) &\implies db = dx \cr }$$ Then write the function and apply the Product rule to find its differential and gradient as $$\eqalign{ y &= Ab \cr dy &= A\,db + dA\,b \cr &= A\,dx - dx\,x^Tb - x\,dx^T\,b \cr &= A\,dx - x^Tb\,dx - xb^T\,dx \cr &= (A - (x^Tb)I - xb^T)\,dx \cr \frac{\partial y}{\partial x} &= A - (x^Tb)I - xb^T \cr }$$