When $A^TA = I$, what matrix is orthogonal to what?

Question

When $A^TA = I$, I am told it is orthogonal. What does that mean?

$A = \begin{bmatrix}cos\theta & & -sin\theta \\ \\ sin\theta & & cos\theta\end{bmatrix}, A^T = \begin{bmatrix}cos\theta & & sin\theta \\ \\ -sin\theta & & cos\theta\end{bmatrix}$

Answer 1

It means that the row vectors (and also the column vectors) form an orthogonal basis, that means if $A$ has dimension $n\times n$, $A$ consists of $n$ linear independent and pairwise orthogonal vectors spanning $\mathbb R_n$

Answer 2

Perhaps the term is suggestive of matrix $\mathbf{A}$ being orthogonal to another matrix $\mathbf{B}$ in some sense like $$\mathbf{A}.\mathbf{B}=0$$ as it would be if $A$ and $B$ were orthogonal vectors. However this is not what it means when considering matrices.

For matrices ìt refers to the set of columns being orthogonal (actually orthonormal) in the usual sense for vectors

$$\mathbf{c_i}.\mathbf{c_j}=\delta_{ij}$$ The same is true for the set of rows.

Because the columns are orthogonal (and rows) the matrix itself is called orthogonal. These matrices have the property you stated.

(Try it with the columns or rows of your 2x2 matrix.)