I am currently having a hard time with the notation for derivatives of spectral functions.
In 2006, Malick and Sendov in (DOI: 10.1007/s11228-005-0005-1) have derived an explicit form for the second derivative of $(1/2)||P_{\mathbb{S}^m_+}(X)||^2$, where $P_{\mathbb{S}^m_+}(X)$ is the metric projection of a symmetric matrix $X$ onto $\mathbb{S}^m_+$, which happens to be gradient of the projection when it is differentiable. I am interested in computing explicitly the directional derivative of the projection in the direction $Y\in \mathbb{S}^m$, say $P'_{\mathbb{S}^m_+}(X,Y)$, using the closed form of the gradient. For instance:
$$\nabla P_{\mathbb{S}^m_+}(X)=U diag^{(12)} (B(X)) U^T,$$
where $$B(X)_{ij}:=\frac{\max\{0,\lambda_i(X)\}-\max\{0,\lambda_j(X)\}}{\lambda_i(X)-\lambda_j(X)}$$ when $i\neq j$ and $B(X)_{ii}:=1$ if $\lambda_i(X)>0$ and $B(X)_{ii}:=0$ if $\lambda_i(X)<0$. They assume $\lambda_1(X)>...>\lambda_n(X)$ and that all of them are nonzero in order for the projection to be differentiable. Also, $diag^{(12)}(B(X))$ inserts the matrix $B(X)$ in the diagonal of a 4-tensor and $U$ is the orthogonal matrix that diagonalizes $X$.
My problem is that $\nabla P_{\mathbb{S}^m_+}(X)$ is a 4-tensor and things like "$\nabla P_{\mathbb{S}^m_+}(X) [Y]$" are not clear to me. I have tried to understand it intuitively because I have no clue of a basic reference in this subject, but I got nowhere. So my questions are:
(1) How can I write $P'_{\mathbb{S}^m_+}(X,Y)$ in terms of $\nabla P_{\mathbb{S}^m_+}(X)$?
(2) Is it possible to write $\nabla P_{\mathbb{S}^m_+}(X)$ in terms of the canonical basis of $\mathbb{S}^m$? If so, how is it exactly?
(3) Could you please point out some basic references on derivatives of matrix valued functions with respect to matrix variables?
I could only find derivatives of matrix valued functions with respect to real variables (D-matrix/D-real) and derivatives of real valued functions with respect to matrices (D-real/D-matrix), but there seems to be nothing about D-matrix/D-matrix.
In Shapiro's paper about directional derivatives of metric projections onto convex sets (http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.570.3090&rep=rep1&type=pdf) he gives directional derivatives of the projection as the solution of an optimization problem where the so-called "sigma-term" appears. However, it does not appear in Malick and Sendov's characterization.
(4) How is the sigma-term related to $\nabla P_{\mathbb{S}^m_+}(X)$?
I appreciate any help, even if it is just a reference indication.