Can you have a matrix of rank 4?

Question

I have been reading this useful guide to matrix ranking: https://www.cliffsnotes.com/study-guides/algebra/linear-algebra/real-euclidean-vector-spaces/the-rank-of-a-matrix

In it, it states "Any collection of more than three 3‐vectors is automatically dependent."

Is this true? I am asking because a lecture on Latent Semantic Indexing stated that the matrix used in the example is of rank 4: https://www.youtube.com/watch?v=CwBn0voJDaw

It also seems a little counter-intuitive, because if I am interpreting this correctly, this would imply in a dataset of hundreds of features, no more than 3 of them are entirely independent of the other features.

If anyone could confirm or deny this for me, and expand on the explanation, that would be excellent.

Answer 1

Sure, you can have a matrix of rank $4$, or $5$ or $6$ or any higher integer. It's just you need longer vectors, spaces of higher dimension than $3$ (indeed the Cliff's notes explicitly state $3$-vectors). You cannot visualise these longer vectors, but they do exist.

In $n$-dimensional space (vectors of length $n$), $n+1$ vectors must be linearly dependent.

Answer 2

The original context is due to

A $3 \times 5$ matrix can be thought of as composed of three $5$‐vectors (the rows) or five $3$‐vectors (the columns). Although three $5$‐vectors could be linearly independent, it is not possible to have five $3$‐vectors that are independent. Any collection of more than three $3$‐vectors is automatically dependent.

The author is discussing about $3 \times 5$ matrix, we have this result that $$A \in \mathbb{R}^{m \times n}, rank(A) \le \min(m, n)$$

Example of rank $4$ matrix is $I_4$ but it has more than $3$ rows and $3$ columns.