Proof that covariance matrix is positive semi-definite (and not positive definite)

Question

The covariance matrix is given $$ \Sigma_{\textbf{XX}} = \text{cov}[\textbf{X}, \textbf{X}] = E[(\textbf{X} - \mu_{\textbf{X}})(\textbf{X} - \mu_{\textbf{X}})^\top] $$ We want to prove that $\Sigma_{\textbf{XX}}$ is positive semi-definite. For this, we can say that if $\Sigma_{\textbf{XX}} \in \mathbb{R}^{n\times n}$, $\forall \textbf{u} \in \mathbb{R}^n$ \begin{align} \textbf{u}^\top \Sigma_{\textbf{XX}} \textbf{u} &=\textbf{u}^\top E[(\textbf{X} - \mu_{\textbf{X}})(\textbf{X} - \mu_{\textbf{X}})^\top]\textbf{u}\\ &= E[\textbf{u}^\top(\textbf{X} - \mu_{\textbf{X}})(\textbf{X} - \mu_{\textbf{X}})^\top\textbf{u}]\\ &= E[((\textbf{X} - \mu_{\textbf{X}})^\top\textbf{u})^\top(\textbf{X} - \mu_{\textbf{X}})^\top\textbf{u}]\quad \textbf{(1)}\\ &= E[((\textbf{X} - \mu_{\textbf{X}})^\top\textbf{u})^2]\ge 0 \quad \textbf{(2)}\\ \end{align}

Here are my 2 questions:

1. What is the linear algebra property allowing to go from $\textbf{(1)}$ to $\textbf{(2)}$?

2. What is the information that allows to state in the last equation that $E[((\textbf{X} - \mu_{\textbf{X}})^\top\textbf{u})^2]\ge 0$ and not $E[((\textbf{X} - \mu_{\textbf{X}})^\top\textbf{u})^2]> 0$ $\iff$ how can we state that $\Sigma_{\textbf{XX}}$ is positive semi-definite and not positive definite?

Edit (thanks to John Hughes) \begin{align} \textbf{u}^\top \Sigma_{\textbf{XX}} \textbf{u} &= E[\underbrace{((\textbf{X} - \mu_{\textbf{X}})^\top\textbf{u})^\top}_{\in \mathbb{R}}\underbrace{(\textbf{X} - \mu_{\textbf{X}})^\top\textbf{u}}_{\in \mathbb{R}}]\\ &= E[((\textbf{X} - \mu_{\textbf{X}})^\top\textbf{u})^2]\ge 0 \\ \end{align}

Answer 1

Look at line 1 as having the form $$ E[s^t s], $$ where $s = (X - \mu_X)^t u$. Now $s^t s$ is, in general, the same as $s \cdot s$, so we can say that $s^t s = \| s \|^2$ (by definition of length!). In this case, it appears that $s$ is just a number, so $s^t s$ is simple $s^2$.

For your second question, look at the number $s$: it might always be zero, in which case $s^2$ would always be zero, so the expected value would always be $0$. When you have a non-negative random variable, the expected value is also non-negative, but not necessarily positive!