Show that $\text{rank}(Df)(A) = \frac{n(n+1)}{2}$ for all $A$ such that $A^TA = I_n$

Question

We identify $\mathbb R^{n \times n}$ with $\mathbb R^{n^2}$ and define $f:\mathbb R^{n^2} \to \mathbb R^{n^2}, A \mapsto A^TA$.

Show that $\text{rank}(Df)(A) = \frac{n(n+1)}{2}$ for all $A$ such that $A^TA = I_n$.

The solution by Omnomnomnom written in a more accessible (for me) way:

We firstly note that $$f(A+H)=(A + H)^T(A + H) = A^TA + H^TA + A^TH +H^TH$$

Consider the following homomorphismus $\Phi:\mathbb R^{n^2} \to \mathbb R^{n^2}, H \mapsto H^TA + A^TH$

Thus the equation above can be written as:

$$f(A+H)= f(A) + \Phi(H) +H^TH$$

Remember that according to the definition $(Df)(A)$ is a homomorphismus from $\mathbb R^{n^2}$ to $\mathbb R^{n^2}$ such that $$\lim_{H\to 0, H \ne 0} \frac{f(A+H)-f(A)-(Df)(A)H}{\|H\|}=0$$

Because $\lim_{H\to 0, H \ne 0}\|\frac{H^TH}{\|H\|}\| \le \lim_{H\to 0, H \ne 0}\frac{\|H^T\|\|H\|}{\|H\|}=\lim_{H\to 0, H \ne 0}\|H^T\|=0$, it follows that:

$$\lim_{H\to 0, H \ne 0} \frac{f(A+H)-f(A)-\Phi (H)}{\|H\|}=\lim_{H\to 0, H \ne 0} \frac{H^TH}{\|H\|}=0$$ and consequently $Df(A)=\Phi$.

Now we note that $\{\Phi(H):H \in \mathbb R^{n^2}\}=\{X\in \mathbb R^{n^2} : X^T = X\}$, because:

1) $(H^TA + A^TH)^T = A^TH^{TT} + H^TA^{TT} = H^TA + A^TH$

2) $S$ is symmetric, then $\Phi(\frac{1}{2}AS)=\frac{1}{2}(SA^TA+A^TAS)=S$

Now it follows: $$\frac{n(n+1)}{2}=\dim \{X\in \mathbb R^{n^2} : X^T = X\}=\dim\{\Phi(H):H \in \mathbb R^{n^2}\}=\text{rank}\Phi=\text{rank}(Df)(A)$$

Answer 1

First, let's compute the derivative as it is defined here, noting that this coincides with the usual definition except that $Df(A)$, rather than producing an explicit $n^2 \times n^2$ matrix, produces a linear map from $\Bbb R^{n \times n}$ to $\Bbb R^{n \times n}$.

In particular, we compute that $$ (A + H)^T(A + H) = A^TA + H^TA + A^TH + o(\|H\|) $$ So, the map we are considering (at any fixed $A \in \Bbb R^{n \times n}$) is the map $H \mapsto H^TA + A^TH$.

Explanatory note: This map is a homomorphism from $\mathbb R^{n\times n}$ to $\mathbb R^{n\times n}$ which is representable by the matrix from $\mathbb R^{n^2 \times n^2}$ , rank of which we are trying to find. To go from this version of the derivative to the usual matrix, it suffices to apply the vectorization operator.

Note, however, that the output of this map is always symmetric. In particular, $$ (H^TA + A^TH)^T = A^TH^{TT} + H^TA^{TT} = H^TA + A^TH $$ Thus, the image of this map is necessarily a subset of the space of symmetric matrices, which is a space of dimension $n(n+1)/2$. Thus, the rank of $Df(A)$ will be at most $n(n+1)/2$.

In fact, there is no better upper bound that applies to all $A$. For example, if we take the derivative at $A = I$, then we find that $$ [Df(I)](H) = H + H^T $$ which is onto the space of symmetric matrices, and therefore has rank exactly equal to $n(n+1)/2$.

---

Proof that $Df(A)$ will have the same rank whenever $A^TA = I$:

We wish to show, in other words, that if $A^TA = I$, then the map $H \mapsto H^TA + A^TH$ is onto. To that effect, it suffices to break this map down into the composition of two maps: $$ T_1(H) = A^TH\\ T_2(H) = H + H^T $$ It is clear that $T_1$ is an isomorphism, with inverse $H \mapsto AH$ (recall that $AA^T = A^TA = I$). We note moreover from our analysis above that $T_2$ is a map with rank $n(n+1)/2$. It follows that the map $$ [Df(A)](H) = [T_2 \circ T_1](H) $$ must also be a linear transformation of rank $n(n+1)/2$, as desired.

Answer 2

A geometrical proof. It suffices to prove that, if $A\in O(n)$, then $dim\ker(Df_A)=n(n-1)/2$. Note that $Df_A(H)=0$ iff $H^TA+A^TH=0$ (Serguey, learn to differentiate a product...), that is, $H$ is in the tangent space $T_A$ to $O(n)$ in $A$. Thus $\ker(Df_A)=T_A$ (it is because $O(n)$ is a level curve of $f$). Since $O(n)$ is a group, $T_A\sim T_I$, the set of skew-symmetric matrices -which is a vector space of dimension $n(n-1)/2$-; and we are done...

EDIT. We can generalize this result; in fact, $O(n)$ has nothing to do here.

Proposition. Let $A\in GL_n(\mathbb{R})$; then $rank(Df_A)=n(n+1)/2$, that is, the function $A\in GL_n(\mathbb{R})\rightarrow A^TA\in S_n^{++}$ is a submersion in each invertible matrix.

Proof. As above, $H\in \ker(Df_A)$ iff $A^TH\in SK_n$, the set of skew-symmetric matrices. Then $\ker(Df_A)=A^{-T}(SK_n)$, a vector space of dimension $n(n-1)/2$ and we are done.