What is the orthogonal complement

Question

Let $\langle A,B \rangle = \text{tr}(B^TA)$. For $M_n(\mathbb{R}$): Find the orthogonal complement of the diagonal-square-matrices.

So I need to find $$U^{\perp} = \left\{ A\in M_n(\mathbb{R}) \mid \forall B\in \{\text{diagonal-square matrices}\}: \langle A,B \rangle = 0 \right\}$$

Hence,
$$\langle A,B \rangle = 0 \iff \text{tr}(B^TA) = 0 \iff \text{tr}(BA) = 0$$

So $A$ is a square matrix and $B$ is every possible square-diagonal matrix. So it must be that $A=0$, isn't it?

Answer 1

It is not the case that $A$ must be $0$. For example, when $n = 2$, the element $$ A = \pmatrix{0&1\\0&0} $$ is a member of $U^\perp$.

Hint: Because $B$ is diagonal, we have $$ \operatorname{trace}(BA) = \sum_{i=1}^n a_{ii}b_{ii} $$ Where $a_{ij}$ and $b_{ij}$ are the entries of $A$ and $B$ respectively. Try different combinations of $b_{ii}$. Note that the above sum says nothing about $a_{ij}$ when $i \neq j$.

Answer 2

Think of this class of matrices as vectors. If

$A = \{x \in \mathbb R^n \mid x_i = 0, \forall i\in I\}$

then

$A^\perp = \{y \in \mathbb R^n \mid x^Ty = 0\} = \{y \in \mathbb R^n \mid y_i = 0, \forall i\not\in I\}$.

(To convince yourself that it has to be for all $i\in I$, just use the indicator vectors for all $i\in I$ as a counterexample. All $e_i, i\in I$ are in $A$, and therefore every element in $B$ must be $0$ for all $i\in I$. But maybe that is trivial.)

In your case, $I$ is merely the set of indices corresponding to the off-diagonal entries. So if

$A = \{X \in \mathbb R^{n\times n} \mid X_{ij} = 0 \forall i\neq j\}$

then

$A^\perp = \{X \in \mathbb R^{n\times n} \mid X_{ij} = 0 \forall i=j\}$.

Note that this set is much bigger than $\{0\}$.