How to compute the derivative of the latent factor where $u_n$, $v_n$ are column vectors. How did he take derivative and obtain the below equation?
I was reading probabilistic matrix factorization where I came across this equation. I don't know how to take vector derivative of another vector. To learn about vector derivatives in general can you point a source?
Note that indexed vectors are really the columns of a matrix, e.g. $$\eqalign{ u_n &= Ue_n \cr v_m &= Ve_m\cr }$$ where $e_k$ denotes the standard vector basis. Similarly, $X$ is also a matrix with columns and components given by $$\eqalign{ x_n &= Xe_n \cr X_{mn} &= e_m^Tx_n = e_m^TXe_n \cr }$$ Now we can write the problem in matrix from $$\eqalign{ 2{\cal L}_n &= \sigma^{-2}(Vu_n-x_n)^T(Vu_n-x_n) + \lambda u_n^Tu_n \cr }$$ at this point the $n$ subscripts are distracting, so let's drop them $$\eqalign{ 2{\cal L} &= \sigma^{-2}(Vu-x)^T(Vu-x) + \lambda u^Tu \cr }$$ and take the differential $$\eqalign{ 2\,d{\cal L} &= 2\sigma^{-2}(Vu-x)^T\,d(Vu-x) + 2\lambda u^T\,du \cr d{\cal L} &= \sigma^{-2}(Vu-x)^TV\,du + \lambda u^T\,du \cr &= \big(\sigma^{-2}(Vu-x)^TV + \lambda u^T\big)\,du \cr }$$ Write down the gradient, set it equal to zero, and solve for the optimal vector $u$ $$\eqalign{ &\frac{\partial{\cal L}}{\partial u} = \sigma^{-2}(Vu-x)^TV + \lambda u^T = 0 \cr &u^TV^TV + \lambda\sigma^2u^T = x^TV \cr &V^TVu + \lambda\sigma^2u = V^Tx \cr &(V^TV + \lambda\sigma^2I)\,u = V^Tx \cr &u = (V^TV + \lambda\sigma^2I)^{-1}V^Tx \cr \cr }$$