Is it true any real matrix add an identity matrix is full rank?

Question

I think the statement is true, since it is how regularization works in linear model (where adding $\lambda I$ to $X^TX$ make it full rank). But why?

Answer 1

$X^TX$ is a symmetric matrix. It is diagonalizable with real eigenvalues.

Let those eigenvalues equal $\lambda_1, \cdots \lambda_n$

If we choose any of the $\lambda_i$ above.

$(X^TX - \lambda_i I)$ will create a singular matrix.

However, $(X^TX - \lambda I)$ where $\lambda \ne \lambda_1, \cdots \lambda_n$

then the eigenvalues of $(X^TX - \lambda I)$ equal $\lambda_1 - \lambda,\cdots,\lambda_n-\lambda$ none of which are equal to $0.$ Hence, $(X^TX - \lambda I)$ is non-singular.

Nearly any perturbation to a singular matrix will make for a non-singular matrix.

Answer 2

For any square matrix $A$ such that $A$ is not invertible, and for all $n\in\mathbb{N}$, $A+\tfrac{1}{n}I$ is in fact invertible. This technique is used to show that invertible matrices are dense in the set square of matrices.

Of course, it is not true for all matrices, as shown by the example $-I$ and $I$.

Many theorems about the general linear group related to this are really interesting. In particular, it is possible to show that $\mathrm{GL}(n)$ has two connected components! You should see Rudin's book on analysis for this.

This is also of interest for numerical analysis since every singular matrix can be approximated by a "close to singular" matrix.

I hope you find this interesting.