I am working with derivatives of the SVD. My setting is the following:
Let $A:(-\varepsilon,\varepsilon)\rightarrow\mathbb{R}^{m\times n}$, $t\mapsto A(t)$ be a differentiable matrix-valued function, $m \leq n$. Assume that $\mathrm{rank}\,A(t)=m$ and the singular values $\sigma_1(t),\dotsc,\sigma_m(t)$ satisfy $\sigma_1(t)>\dotsb>\sigma_m(t)$ for all $t$.
Now we look at the "economy-size" SVD $$A(t)=U(t)\varSigma(t)V(t)^\mathrm{T},$$ $U(t) \in \mathbb{R}^{m\times m}$, $\varSigma(t) \in \mathbb{R}^{m\times m}$, $V(t) \in \mathbb{R}^{n\times m}$. By construction, this decomposition is unique up to a simultaneous change of sings in a column of $U(t)$ and the corresponding column of $V(t)$. Using the product rule, we get $$\dot{A}(t)=\dot{U}(t)\varSigma(t)V(t)^\mathrm{T} +U(t)\dot{\varSigma}(t)V(t)^\mathrm{T} +U(t)\varSigma(t)\dot{V}(t)^\mathrm{T},$$ where the dot denotes the derivative with respect to $t$.
These two papers discuss the calculation of the matrices $\dot{U}(t)$, $\dot{\varSigma}(t)$ and $\dot{V}(t)$:
http://www.ics.forth.gr/_publications/2000_eccv_SVD_jacobian.pdf (Section 2)
http://people.maths.ox.ac.uk/gilesm/files/NA-08-01.pdf (Section 3.2)
Both papers describe how these matrices can be obtained form the SVD $A(t)=U(t)\varSigma(t)V(t)^\mathrm{T}$ and the derivative $\dot{A}(t)$.
However, I am not interested in a full SVD, but in a truncated SVD, using only the $r$ dominant singular values and corresponding singular vectors, and in its derivative. I would like to avoid computing a full SVD of $A(t)$ and to compute the derivatives only using a truncated SVD. The methods described in the two papers above can be easily modified to compute the first $r$ diagonal entries of $\dot{\varSigma}(t)$ only using a truncated SVD. However, this does not work for the dominant singular vectors.
Does anyone know a method for computing the variations of the dominant singular vectors without computing the full SVD? I have come to believe this might be impossible because we need the information from the variation in the non-dominant singular vectors to compute these derivatives but I could not find a proof for this, either. Any thoughts on this?
Thanks a lot!
I don't know if you got the answer you wanted but this paper might answer the question:
https://arxiv.org/abs/1705.08521
It is shown that the truncated SVD of order $r$ is differentiable at a matrix $R$ if the singular values of order $r$ and $r+1$ are distinct and the formula for the derivative is given.