For $M\succeq 0$ and $M\in \mathbb{R}^{n\times n}$, we know the following facts:
- $$\lambda_{\min} = \min_{\|x\|=1} x^TMx$$
- $$\lambda_{\max} = \max_{\|x\|=1} x^TMx$$
My questions are the following:
1. Do the above facts hold for any square matrices $M\in \mathbb{R}^{n\times n}$, or does this matrix have to be at least symmetric?
2. Are the above facts equivalent to "$\|x\|=1$, $x\in \text{span}(M)$"? i.e., we restrict the condition $x\in \mathbb{R}^n$ to $x\in \text{span}(M)$
where $\text{span}(M)$ represents the subspace expanded by the column vectors of $M$.
The first question does not have an answer since an arbitrary matrix can possess nonreal eigenvalues and equations (1) and (2) above are meaningless in this context.
However, even if we adjust the question, the answer is 'no'. Given $A \in M_n(\mathbb{C})$, the field (of values) of $A$ (also called the numerical range of $A$), is defined by $$ F(A) = \{ z^*Az\mid z^*z = 1\}. $$ It is well-known that $F(A)$ is compact (easy to prove), $\sigma(A) \subseteq F(A)$ (easy to prove), and convex (nontrivial; known as the Hausdorff-Toeplitz theorem).
For $A \in M_n(\mathbb{C})$, let $\rho_{\min}(A) := \min\{\lambda \mid \lambda \in \sigma(A)\}$ and $\rho_\max(A) := \max \{\lambda \mid \lambda \in \sigma(A)\}$. Is it the case that \begin{equation} \rho_\min(A) = \min\{ |x| \mid x \in F(A) \} \tag{$\ast$} \end{equation} and \begin{equation} \rho_\max(A) = \max\{ |x| \mid x \in F(A) \}? \tag{$\ast\ast$} \end{equation}
The answer is 'no'. The elliptical range theorem states that the field of a two-by-two matrix is a (possibly degenerate) elliptical disk. In particular, if $$ A = \begin{bmatrix} a & b \\ 0 & c \end{bmatrix}, $$ then $F(A)$ is an elliptical disk with foci at $a$ and $c$ and minor-axis length $|b|$.
For example, the field of the matrix $$ \begin{bmatrix} 1 & i \\ 0 & -1 \end{bmatrix} $$ is given by:
From the picture, it is clear that $(\ast)$ and $(\ast\ast)$ do not hold.