show that $M \mapsto \max( \{ \operatorname{Tr}(OM) \mid O \in O_n ( \mathbb R )\})$ with is continuous

Question

\begin{align} f : M_n (\mathbb R) & \to \mathbb R \\ M & \mapsto \max(\{\operatorname{Tr}(OM) \mid O \in O_n ( \mathbb R )\}) \end{align}

prove that $f$ is well defined and continuous

Let $M\in M_n (\mathbb R)$

\begin{align} h : O_n ( \mathbb R )& \to \mathbb R \\ M & \mapsto \operatorname{Tr}(OM) \\ \end{align}

$h$ is continuous (continuity of matrix multiplication and of Trace) and $O_n ( \mathbb R )$ is a compact space so $\max(\{\operatorname{Tr}(OM) \mid O \in O_n ( \mathbb R )\})$ is well defined ie $f$ is well defined.

To prove that $f$ is continuous, I tried to show that $f$ is linear but $f$ isn't linear, and I am stuck.

Answer 1

$$(A|B):=trace(A^TB)$$

is a dot product (see here) on the space of $n \times n$ matrices.

Here it doesn't matter that the $^T$ operator is absent because the transpose of an orthogonal matrix is also an orthogonal matrix.

Therefore, you just have to invoke the continuity of dot product.