Can we define component of a matrix which is orthogonal to another matrix?

Question

Given two vectors $A$ and $B$ one can easily find component of $A$ along $B$ and component of $A$ perpendicular/orthogonal to $B$ and vice versa. This is possible as we can define dot product of two vectors. As I understand if $A$ and $B$ are matrix then our dot product does not make sense, does it? If not is there a way to define component and orthogonal component of a matrix with respect to the other? Or there is no such concepts for matrices?

Answer 1

The Frobenius inner product of matrices $A, B \in \mathbb R^{m \times n}$ is defined by

$$\langle A, B\rangle := \mbox{tr} (A^T B) = \mbox{tr} (B A^T) = \mbox{tr} (A B^T) = \mbox{tr} (B^T A)$$

For example, the standard basis "vectors" for $\mathbb R^{2 \times 2}$ are

$$M_{11} := \begin{bmatrix} 1 & 0\\ 0 & 0\end{bmatrix} \qquad \quad M_{12} := \begin{bmatrix} 0 & 1\\ 0 & 0\end{bmatrix} \qquad \quad M_{21} := \begin{bmatrix} 0 & 0\\ 1 & 0\end{bmatrix} \qquad \quad M_{22} := \begin{bmatrix} 0 & 0\\ 0 & 1\end{bmatrix}$$

Note that

$$\langle M_{11}, M_{11}\rangle = 1 \qquad \qquad \langle M_{11}, M_{12}\rangle = 0 \qquad \qquad \langle M_{11}, M_{21}\rangle = 0 \qquad \qquad \langle M_{11}, M_{22}\rangle = 0$$

$$\langle M_{12}, M_{11}\rangle = 0 \qquad \qquad \langle M_{12}, M_{12}\rangle = 1 \qquad \qquad \langle M_{12}, M_{21}\rangle = 0 \qquad \qquad \langle M_{12}, M_{22}\rangle = 0$$

$$\langle M_{21}, M_{11}\rangle = 0 \qquad \qquad \langle M_{21}, M_{12}\rangle = 0 \qquad \qquad \langle M_{21}, M_{21}\rangle = 1 \qquad \qquad \langle M_{21}, M_{22}\rangle = 0$$

$$\langle M_{22}, M_{11}\rangle = 0 \qquad \qquad \langle M_{22}, M_{12}\rangle = 0 \qquad \qquad \langle M_{22}, M_{21}\rangle = 0 \qquad \qquad \langle M_{22}, M_{22}\rangle = 1$$

Answer 2

There is such a thing called orthogonal matrix in linear algebra. An orthogonal matrix is a square matrix with real entries whose columns and rows are orthogonal unit vectors: $$AA^T=I$$ Each entry is a dot product of a column of $A$ with an other column of $A$.
Orthogonal matrices preserve dot products, the length of vectors as well as the angles between them: $Ax\cdot Ay=x\cdot y$