Rank of Principal submatrix of annihilator matrix

Question

In the classic linear regression setting,

$y = X\beta+\epsilon$, where $y\in \mathbb{R}^n$, $X\in \mathbb{R}^{n\times p}$, $\beta\in\mathbb{R}^p$, and $\epsilon\sim N(0,\sigma^2I)$.

$P = X(X^\top X)^{-1}X^\top$ is called the projection matrix or hat matrix.

$M = I-P$ is called the annihilator matrix.

We know if $rank(X) = p$, then $rank(P) = p$ and $rank(M) = n-p$.

Suppose $X$ indeed has full column rank. In particular, let's assume that the rows of X are in general positions, i.e. any p rows of X are linearly independent.

I want to find an upper bound $S$, s.t. for any $s\leq S$, any $s\times s$ principal submatrix of M is invertible.

A few useful properties:

1. $M$ eigen spectrum contains p zeros and n-p ones.

2. $P$ is a projection matrix in the sense that $P^2=P$. Similarly, $M^2=M$.

Any hints are helpful.

Answer 1

Unfortunately, there are matrices $X$ (with full rank) such that for any $s \geq 1$, there is a non-invertible $s \times s$ submatrix. For instance: $$ X = \pmatrix{1\\0\\0} \implies M = \pmatrix{0&0&0\\0&1&0\\0&0&1} $$ Note that any principal submatrix which includes the first row/column will fail to be invertible.