Why diagonal matrix SVD sorted from largest to smallest value?

Question

Why diagonal matrix SVD sorted from largest to smallest value?

D is diagonal matrix, $D=(d_1 \ge ,d_2 \ge ,..., \ge d_L)$.

Whether there is a journal that could explain this?

Answer 1

There is no reason other than convention. You can order the eigenvalues anyway you want, as long as you order the rows and columns of $U$ and $V$ accordingly.

The reason this is the convention, is that SVD is mostly used in applications for finding the least squared solution, given by the eigenvector corresponding to the smallest eigenvalue. Since developers don't want to perform a costly sort to find the smallest eigenvalues every time, most implementations output sorted SVD matrices.

A simple proof this is so:

Note that if $A=U\Sigma V$ the element $a_{ij}$ is: $$a_{ij} = \sum_k \sigma_k u_{ik}v_{kj}$$ Note that the order of the sum over $k$ doesn't matter, as long as you reorder the rows of $U$, the columns of $V$ and the eigenvalues of $\Sigma$ in the same order.

Answer 2

I think that the purpose behind sorting the singular value is to show the maximum and minimum energy sensed in direction of the left or right singular vector. but the question is this singular values came from where, is it the eigen value of (A.A Transpose) or from (A Transpose. A) and this value corresponds to which column vector from the original matrix A.