I have several derivatives need to be calculated. I was wondering whether it's correct or not.
Assume $x\in \mathbb{R}^{n \times 1}$, then
- matrix-vector: $$\frac{\partial x x^\top}{\partial x} = ? $$
- scalar-vector: $$\frac{\partial x^\top x}{\partial x} = ? $$
- vector-vector: $$\frac{\partial x x^\top x}{\partial x} =? $$
- vector-vector: $$\frac{\partial x^\top x x^\top}{\partial x} =? $$
- vector-vector: $$\frac{\partial x x^\top x}{\partial x^\top} =? $$
- vector-vector: $$\frac{\partial x^\top x x^\top}{\partial x^\top} =? $$
I take a look at the matrix cookbook, but still get confused about this examples. Could anyone provide detailed steps to get the results by applying the product rule?
Start with the scalar-vector case. Write the function in terms of the inner product, then find the differential and then the gradient. $$\eqalign{ \phi &= x^Tx = x\cdot x \cr d\phi &= 2x\cdot dx \cr \nabla_x\phi &= 2x \cr\cr }$$ Next up is the vector-vector case. $$\eqalign{ y &= xx^Tx = x\phi \cr dy &= \phi\,dx + x\,d\phi &= (\phi\,I + 2xx)\cdot dx \cr \nabla_x y &= \phi\,I + 2xx^T \cr\cr }$$ Next is the matrix-vector case, but I'm not sure how to handle that since the result is a 3rd order tensor.