Elsewhere on this site, I found a very compact proof of the Gauss-Markov theorem, seen below. I don't understand the justification for the middle step with the inequality. Specifically, what property of the projection matrix $P_x$ allows us to conclude that $\sigma^2CC' \geq \sigma^2CP_xC'$ ?
On
In the referenced proof, the inequality $CC'\ge CP_XC'$ involves two matrices, hence it asserts that $CC'-CP_XC'$ is positive semidefinite.
Why is this assertion true? Note that $P_X$ is symmetric and idempotent, therefore so is $Q:=I-P_X$, so in particular $Q=Q'Q$. To show $CC'-CP_XC'=C(I-P_X)C'=CQC'$ is positive semidefinite, we must verify $v'(CQC')v\ge 0$ for each vector $v$. Indeed, $$v'(CQC')v= (C'v)'Q(C'v)=(C'v)'(Q'Q)(C'v)=(QC'v)'(QC'v)\ge 0.$$
I am slightly unclear on the dimensions of your objects, so I am going to assume for simplicity that $C$ is a row vector. One can probably generalize this argument if this assumption is incorrect.
Let $P$ be orthogonal projection onto some subspace, and $Q=I-P$ be projection onto the orthogonal complement of $X$. Then $C'=P(C')+Q(C')$ where $P(C')$ and $Q(C')$ are orthogonal to each other. By the Pythagorean theorem, $||C||^2=||P(C')||^2+||Q(C')||^2$, but both terms on the right hand side are non-negative, so in particular, we have $||C||^2\geq ||P(C')||^2.$
Also, recall that an orthogonal projection matrix is idempotent ($P^2=P$) and symmetric.
Now, we have that $CC'=||C||^2$, but $CPC'=(CP)(PC')=(PC')'(PC')=||PC'||^2$.
A more general but less geometric result is that if $A$ is symmetric, and $v$ is a column vector, then $v^TAv \leq \lambda_{max}||v||^2$, where $\lambda_{max}$ is the largest eigenvalue of $A$. To see this, by the spectral theorem, we have an orthonormal eigenbasis, $u_1, \ldots u_n$, with $Au_i=\lambda_i u_i$. If $v=\sum a_i u_i$, then one can compute $v^T Av= \sum \lambda_i a_i^2 \geq \sum \lambda_{max} a_i^2 =\lambda_{max}||v||^2$.
The result for projections now follow from the fact that projections are symmetric with eigenvalues $0$ and $1$.