I am wondering if its possible to obtain an analytic expression for the gradient of $$f(B) = (A - B)\left[(A - B)'(A - B)\right]^{-\frac{1}{2}}$$ with respect to $B$ where $A \in \mathbb{R}^{s \times t}$, $B \in \mathbb{R}^{s \times t},$ and $(A - B)$ has rank $t$ and $t$ distinct singular values.
Trying to vectorize the expression (since gradient of vector with respect to vector has analytic form): $$ {\rm vec}\left(A\left[(A - B)'(A - B)\right]^{-\frac{1}{2}}\right) - \left(\left[(A - B)'(A - B)\right]^{-\frac{1}{2}} \otimes I\right) {\rm vec}(B),$$ but since $B$ cannot be vectorized inside the square root, I'm not sure how (or if its possible to) proceed.
Any insight would be appreciated!
Define a new variable $$\eqalign{ X &= B-A \quad\implies dX = dB \cr }$$ Write the function in terms of this new matrix. $$\eqalign{ F &= -X(X^TX)^{-1/2} \cr }$$ Multiply each side by its transpose and calculate their differentials. $$\eqalign{ FF^T &= X(X^TX)^{-1}X^T = XX^+ \cr dF\,F^T+F\,dF^T &= dX\,X^+ + X\,dX^+ \cr &= dX\,X^+ + X^{+T}dX^T(I-XX^+) - XX^+dX\,X^+ \cr &= (I-XX^+)\,dX\,X^+ + X^{+T}dX^T(I-XX^+) \cr &= P\,\,dX\,X^+ + X^{+T}dX^TP \cr }$$ where $X^+$ denotes the pseudoinverse, and $P$ is an orthogonal projector into the nullspace of $X$.
Vectorize and solve for the gradient. $$\eqalign{ \Big((F\otimes I) + (I\otimes F)K\Big)\,{\rm vec}(dF) &= \Big((X^{+T}\otimes P) + (P\otimes X^{+T})K\Big)\,{\rm vec}(dB) \cr M\,df &= N\,db \cr \frac{\partial f}{\partial b} &= M^+N \cr }$$ where $K$ is the commutation matrix associated with the Kronecker product.