Reference Request: Does $\lambda_\min\|x\|^2 \leq x^TPx \leq \lambda_\max\|x\|^2$ hold for positive semidefinite matrix?

Question

From Chong and Zak's optimization text:

Rayleigh's inequality: if an $n \times n$ matrix $P$ is real symmetric positive definite then

$$\lambda_\min\|x\|^2 \leq x^TPx \leq \lambda_\max\|x\|^2$$

Does the above inequality still hold if $P$ is positive semidefinite instead? Can someone cite a reference that would be very helpful!

Answer 1

If $P$ is real and symmetric then the eigenvalues are real and there is an orthogonal matrix (of eigenvectors) $Q$ such that $Q^T P Q = \Lambda$, where $\Lambda$ is a diagonal matrix of eigenvalues.

Then $\langle x, P x \rangle = \langle Q^T x, \Lambda Q^T x \rangle = \sum_k \lambda_k [Q^Tx]_k^2$ and so $\lambda_\min \|x\|^2 = \lambda_\min \|Q^Tx\|^2 = \sum_k \lambda_\min [Q^Tx]_k^2 \le \sum_k \lambda_k [Q^Tx]_k^2 = \langle x, P x \rangle$, and similarly for the $\max$ eigenvalue, mutatis mutandis.