Normal space of orthogonal matrix.

Question

Let $ O(n) $ be the manifold of orthornormal matrix, i.e. $$ O(n)=\{A\in\mathbb{R}^{n\times n}:A^TA=I\}. $$ Then $ O(n) $ is a submanifold of $ \mathbb{R}^{n\times n} $. On $ O(n) $, there is a Riemannian metric on $ O(n) $ induced by Euclidian metric of $ \mathbb{R}^{n\times n} $. A well-known result shows that for $ A\in O(n) $, $$ T_AO(n)=\{X\in\mathbb{R}^{n\times n}:X^{T}A+A^TX=0\}, $$ where $ T_AO(n) $ denotes the tangent space of $ O(n) $ at the point $ A $. I want to calculate the normal space of $ O(n) $ at the point $ A $. I know that formally, the normal space can be seen as $$ N_AO(n)=\{Y\in\mathbb{R}^{n\times n}:X:Y=0,\,\,\forall X\in\mathbb{R}^{n\times n}\text{ such that }X^TA+A^TX=0\}, $$ where $ X:Y=\sum_{i,j=1}^nx_{ij}y_{ij} $ with $ X=(x_{ij}) $ and $ Y=(y_{ij}) $. I want to simplify this but I cannot go on. Can you give me some hints or references ?

Answer 1

Note you can also describe $T_A O(n)$ as $$ T_A O(n) = A \cdot T_I O(n) = \{X\in\mathbb{R}^{n\times n}|X=AY, \text{ for some } Y + Y^T = 0\}.$$

It's not too hard to see that the normal space at the origin is $N_I O(n) = \{Y\in\mathbb{R}^{n\times n}| Y - Y^T = 0\}$ and since our inner product is invariant under left multiplication: $AX:AY = X:Y$ we have $$N_A O(n) = A\cdot N_I O(n)= \{X\in\mathbb{R}^{n\times n}|X=AY, \text{ for some } Y - Y^T = 0\} = \{X\in\mathbb{R}^{n\times n}|X^TA - A^TX =0\}.$$

Answer 2

Start by defining the tangent space $ T_AO(n) $ like you said.

$$T_AO(n)=\{X\in\mathbb{R}^{n\times n}:X^{T}A+A^TX=0\}$$

With the elements of $ T_AO(n) $ being the tangent vectors to $O(n)$ at the point $A$.

Now you need to determine all of the vectors $ U \in \mathbb{R}^{n\times n} $ which satisfy $X:U = 0 \ \forall \ X \ \in T_AO(n) $ being $:$ the Frobenius inner product.

This condition can also be expressed more cleanly as

$$T_r(X^TU) = 0$$

being $ T^r $ the trace of a matrix. You can now see that there exists a set of matrixes $X^TA + A^TX \ \forall \ X \ \in \mathbb{R}^{n \times n}$ also forms a vector subspace, which we can call $S$, so that all matrices in $T_AO(n)$ lie in $S$.

To find the normal space we must calculate the ortogonal complement of $S$.

$$S⊥ \ = \{{ U \in \ \mathbb{R}^{n\times n}: T_r(X^TY) = 0, \ \forall \ X \in S}\}$$