Derivative of 2-norm of a matrix.

Question

I have a function: $$f(A)=cond(AB)=\left \|{AB}\right \|_2 \left \| {(AB)^{-1}} \right \|_2 $$ We know that: $$\left \| X \right \|_2= \sqrt{\lambda_{max}(X^TX)}$$ So: $$f(A)=\sqrt{\dfrac {\lambda_{max}((AB)^T(AB))} {\lambda_{min}((AB)^T(AB))}}$$ How should I take the derivation of $f(A)$ with respect to matrix $A$ ? $$\dfrac {df(A)} {dA}$$

Answer 1

$ \def\k{\kappa} \def\s{\sigma} \def\e{\varepsilon} \def\o{{\tt1}} \def\so{\s_\o} \def\sr{\s_r} \def\uo{u_\o} \def\ur{u_r} \def\vo{v_\o} \def\vr{v_r} \def\LR#1{\left(#1\right)} \def\op#1{\operatorname{#1}} \def\trace#1{\op{Tr}\LR{#1}} \def\rank#1{\op{rank}\LR{#1}} \def\cond#1{\op{cond}\LR{#1}} \def\frob#1{\left\| #1 \right\|_F} \def\qiq{\quad\implies\quad} \def\p{\partial} \def\grad#1#2{\frac{\p #1}{\p #2}} \def\c#1{\color{red}{#1}} \def\CLR#1{\c{\LR{#1}}} \def\fracLR#1#2{\LR{\frac{#1}{#2}}} $Let $C=AB$ and calculate its SVD and condition number $$\eqalign{ r &= \rank C,\qquad C &= \sum_{i=\o}^r \s_i u_i v_i^T \qquad \k = \cond C &= \fracLR{\so}{\sr} \\ }$$ From this post, the gradient of the $k^{th}$ singular value is $\;{\large\grad{\s_k}C} = u_kv_k^T\;\;$ (if it exists)

The gradient of the condition number (assuming it's finite) is therefore $$\eqalign{ \grad{\k}C &= \fracLR{\sr\uo\vo^T-\so\ur\vr^T}{\sr^2} &= \frac{\uo\vo^T-\k\ur\vr^T}{\sr} \\ }$$ Rearrange this to recover the gradient wrt $A$ $$\eqalign{ d\k &= \fracLR{\uo\vo^T-\k\ur\vr^T}{\sr}:\c{dC} \\ &= \fracLR{\uo\vo^T-\k\ur\vr^T}{\sr}:\CLR{dA\:B} \\ &= \fracLR{\uo\vo^T-\k\ur\vr^T}{\sr} B^T:dA \\ \grad{\k}A &= \fracLR{\uo\vo^T-\k\ur\vr^T}{\sr}B^T \\ }$$

---

In the above derivation, a colon denotes the Frobenius product $$\eqalign{ M:B &= \sum_{i=1}^m\sum_{j=1}^n M_{ij}B_{ij} \;=\; \trace{M^TB} \\ B:B &= \frob{B}^2 \qquad \{ {\rm Frobenius\;norm} \}\\ M:B &= B:M \;=\; B^T:M^T \\ \LR{XY}:B &= X:\LR{BY^T} \;=\; Y:\LR{X^TB} \\ }$$