I am trying to implement the derivative of the diagonalization a positive-definite matrix of size $n\times n$ wrt its $n(n+1)/2$ lower (or upper) components. Just so that we agree on notations, I look for a decomposition of the shape $M = UDU^T$, with $U^TU =I_n$ and $D$ is diagonal (with positive diagonal terms since M is p.d.), such as the output of the eigen() function from R. Thank you in advance !
Edit: I found a solution. I think that it may make scream people who do real maths, but I managed to confirm it numerically. Feel free to point if something seems wrong anyway.
To get the derivative of the eigenvalues, let $U_i$ be the $i^{th}$ column of U, and $D_i$ the corresponding eigenvalue. Let $M = \Sigma M_j e_j$, $(e_1,\ldots , e_{d(d+1)/2})$ being a basis of symmetric matrices. Start from $U_i^T(M-D_iI)U_i = 0$. Differentiate to obtain $\partial U_i^T(M-D_iI)U_i/\partial M_{j} = 0$. Then, apply the product rule and cancel some stuff to obtain
$$ \partial D_i /\partial M_j= U_i e_j U_i^T $$
To get the eigenvectors, start from $(M-D_iI)U_i = 0$ and get $\partial (M-D_iI)U_i/\partial M_{j} = 0$. Then, applying the product rule, you get $$ -(e_j - I \partial D_i /\partial M_j)U_i = (M-D_iI)\partial U_i/\partial M_{j}.$$ Here, it looks like you are screwed, because $(M-D_iI)$ is not invertible. But, because of the fact that $U^tU = I$, you will have $<\partial U_i/\partial M_{j},U_i>$ = 0. Then, the when multiplying $(M-D_iI)$, you lose one component that is always zero. Then you can go ahead and get $$ -pseudoinv(M-D_iI)(e_j - I \partial D_i /\partial M_j)U_i = \partial U_i/\partial M_{j}.$$