I'm reading about applications of SVD in data compression and they have the following blurb...
Consider some matrix A with rank five hundred; that is, the columns of this matrix span a 500-dimensional space. Encoding this matrix on a computer is going to take quite a lot of memory! We might be interested in approximating this matrix with one of lower rank - how close can we get to this matrix if we only approximate it as a matrix with rank one hundred, so that we only have to store a hundred columns? What if we use a matrix of rank twenty? Can we summarize all of the information in this very dense, 500-rank matrix with only a rank twenty matrix?
If $A$ is the matrix of some grayscale image, where each entry is a decimal between $0$ and $1$, how can we guarantee that this matrix is full rank to begin with? ie. how do we know that this image has all linearly independent columns? For example, I could construct an image that is white (all the entries of the matrix are $1$). This would be a rank $1$ matrix, but would still be very expensive to store. However, it does make sense to talk about the rank of $\Sigma$ where $A = U\Sigma V^T$, since $\Sigma$ is a diagonal matrix.
Thanks for any clarification.
There is no requirement that the matrix to be compressed is full rank to begin with. If it has rank one, as per your example, the SVD will reveal that.