on the Singular Value Decomposition

Question

If $T$ is a self-adjoint linear map on a $n$-dimensional inner product space $X$ (either real or complex) then we know by the spectral theorem that there is an orthonormal basis of $X$, call it $v = \{v_1, v_2, \dots, v_n\}$, under which the matrix of $T$ is diagonal.

Even though this is not true for arbitrary linear maps, we can still do something. In fact, by the singular value decomposition, $T$ is still guaranteed to have a diagonal matrix if we are willing to use two different orthonormal bases for $X$, call them $e = \{e_1, e_2, \dots, e_n\}$ and $f = \{f_1, f_2, \dots, f_n\}$.

At first, I thought that if we performed a SVD to a self-adjoint operator, then we would get, up to a reordering, $v = e = f$ and the same diagonal matrices.

Actually this is not true, because the SVD is even stronger that that: the elements of the diagonal matrix are positive!

So I am left wondering if there exists a less strong SVD where the elements of the diagonal matrix are not constrained to be positive, so that if we performed this decomposition to a self-adjoint operator we would get the same of an eigenvalue-eigenvector decomposition..

Answer 1

I think the question turns on how much information the singular values $\sigma_i$ carry for an operator which may not be normal.

Suppose we want to factor $T = U \Gamma V^*$, and we're allowed to choose $\Gamma$ diagonal with non-positive (or even non-real) entries. The entries would have to have moduli $|\gamma_i | = \sigma _i$ in some ordering, by an argument I'll give below. And beyond that, I'm not sure what kind of canonical choice we could make for the arguments of the $\gamma_i$.

We know what $U$ and $V$ have to be: if $T: W \to W$ is a complex linear operator and $T = U \Gamma V^*$ for $\Gamma$ diagonal, then $T^*T = V \Gamma^* \Gamma V^*$. This has to be the orthogonal diagonalization of the positive operator $T^* T$, so $V$ must be the eigenvectors for $T^* T$. By a similar argument $U$ must be the eigenvectors for $TT^*$. And we can see that $\overline {\gamma_i}{\gamma_i} = |\gamma_i|^2 $ have to be the eigenvalues of $T^* T$, so we know the modulus of $\gamma_i$. But which argument for $\gamma_i$ do we choose, if not zero? In order to get what you're asking for, we'd like to make some choice that reflects something about the original operator $T$ (like, say, something to do with its eigenvalues). But I'm not sure what that would be.

Answer 2

Let $T = U \Sigma V^*$ be an SVD of $T$. Then

$$U \Sigma V^* = T = T^* = V \Sigma U^*,$$

i.e.,

$$\Sigma = (U^*V) \Sigma (U^*V)^*.$$

Obviously, $U^*V$ can be a diagonal, in which case it is trivial to change signs in $\Sigma$ in a way that $U$ and $V$ become equal.

But if, for example, $T$ has two same singular values, i.e., $\Sigma$ has two same diagonal elements, then $U^*V$ may be a permutation (maybe also multiplied by a unitary diagonal), so the fix is harder.

Even worse, if $T$ is singular or rank $r$, then

$$T = \begin{bmatrix} U_1 & U_2 \end{bmatrix} \begin{bmatrix} \Sigma_1 \\ & 0 \end{bmatrix} = \begin{bmatrix} V_1^* \\ V_2^* \end{bmatrix} = \begin{bmatrix} U_1 & 0 \end{bmatrix} \begin{bmatrix} \Sigma_1 \\ & 0 \end{bmatrix} = \begin{bmatrix} V_1^* \\ 0 \end{bmatrix} = U_1 \Sigma V_1^*.$$

Here, $U_2$ and $V_2$ are pretty arbitrary (their columns still have to be normal, and orthogonal to each other and those of $U_1$ and $V_1$, respectively), which makes them quite unrelated. However, this can be fixed by computing some form of reduced SVD (i.e., computing only rectangular orthonormal $U_1$ and $V_1$).

So, changing the SVD to behave the way you want is not all that trivial. I think it is much easier to check if the matrix is Hermitian/symmetric and, if it is, compute Schur decomposition instead of the SVD. The check may not be as simple if you want more generality ($\Sigma$ not necessarily real, i.e., $T$ any normal matrix).