In order to compare two matrices, suppose A and B, based on the level of linear dependency between column vectors of a given matrix, I can think about following measures:
- Rank of a matrix: If the rank of matrix A is higher than the rank of matrix B, this implies that the matrix B has more dependent columns than matrix A.
The measure based on the rank of a matrix works well if few of the singular values of a matrix are close to zero. This is not an appropriate measure if singular values do not decay to zero or slowly decay to zero for example for an ill-conditioned matrix.
- The determinant of a matrix: The determinant of a matrix signifies the volume spanned by the columns of a matrix. The higher the volume, more independent the vectors are (assuming each column is normalized).
The measure based on the determinant of a matrix seems to appropriate for ill-conditioned matrix where determinant is simply the product of eigenvalues.
- Now, a third measure based on intuition, i.e., the decay rate of singular values: Faster the decay, more dependent are the columns of a matrix.
Suppose the largest singular values of matrix A and matrix B are same. If the singular value decay of matrix B is faster than of the matrix A. The product of singular values for matrix B is lower than for the matrix A. This is equivalent to say that the determinant of matrix B is lower than the matrix A and therefore columns of the matrix B are more dependent than the matrix A.
But, I can not come up with a proper mathematical justification for the claimed third measure for any two matrices.
Does the third measure make sense to you for a general case? Can you think of a case where the third measure fails?