Dimension of the group of all motions in $\mathbb{R}^n$ which leaves a fixed r-plane invariant.

Question

As the title, I would like to ask the dimension of the group of all motions in $\mathbb{R}^n$ which leaves a fixed r-plane $L^0_r$ invariant.

Here is my observation, but I don't know if it is useful to help solving the problem.

(1) Since the group of motions $M_n$ in $\mathbb{R}^n$ consists of the group of rotations, $SO(n)$, and the group of translations which is isomorphic to $\mathbb{R}^n$, we have $$M_n= SO(n)\times \mathbb{R}^n, $$ and $$dim(M_n)=\frac{n(n-1)}{2}+n=\frac{n^2+n}{2}.$$

(2) Given a fixed r-plane $L^0_r$ in $\mathbb{R}^n$ ($L^0_r$ may as a subspace, passes through the origin $O$ or not (as an affine subspace of $\mathbb{R}^n$). Let $h_r$ be the subgroup of $M_n$ such that any element in $h_r$ leaves $L^0_r$ invariant. By the fundamental knowledge of homogeneous spaces, we have the fact that there is an one-to-one correspondence between the space of r-planes $L_r$ in $\mathbb{R}^n$ and the homogeneous space $M_n/h_r$ by $$L_r=g \cdot L^0_r\leftrightarrow g\in M_n/h_r,$$ where $g\cdot L^0_r$ is the left translation in $\mathbb{R}^n$ regarded as a Lie group $(\mathbb{R}^n, \cdot)$.

Answer 1

First of all your group $M_n$ of motions is not a direct product as you wrote. The factors do not commute. However, the translations form a normal subgroup so $M_n$ is a semi-direct product.

Now you already know the dimension on the motion group $M_r$ so it remains to determine what group keeps the (affine) subspace pointwise fixed and the dimension of that group.

Answer 2

Suppose $\Pi\subseteq\mathbb{R}^n$ is an $r$-dimensional affine subspace. There are two interpretations I can think of to a rigid motion "fixing $\Pi$": fixing it pointwise and fixing it setwise.

The key here is that $\mathrm{Stab}(hx)=h\mathrm{Stab}(x)h^{-1}$ for any group action. (Since pointwise stabilizers of sets are intersections of point-stabilizers, i.e. $\bigcap_{x\in X}\mathrm{Stab}(x)$, this is true for sets as well.)

Pointwise. Let $G$ be the pointwise stabilizer of $\Pi$. If $h\in G$ is any rigid motion for which $\Pi=h\Lambda$ for a linear subspace $\Lambda$, then $G$ is conjugate to $\Lambda$'s pointwise stabilizer by $h$. (Such an $h$ always exists: translation by a chosen vector in $\Pi$.) So without loss of generality, let's deal with $\Lambda$.

Note $g\in G$ fixes $0\in\Lambda$ so it's linear. We can write $\mathbb{R}^n=\Lambda\oplus\Lambda^{\perp}$ and $g=\mathrm{Id}_{\Lambda}\oplus R$ where $R$ is some rotation of $\Lambda^{\perp}$, so $G\cong\mathrm{SO}(\Lambda^{\perp})\cong \mathrm{SO}(n-r)$.

Setwise. This means $g\Pi=\Pi$ for all $g\in G$, but $g$ can act nontrivially on $\Pi$ itself, i.e. $gx=x$ doesn't necessarily hold for $x\in\Pi$. The same idea applies as before, where we can instead consider a linear subspace $\Lambda$, but now $g$ may not fix $0$. However, if $h$ is translation by the vector $g0$, then we have $g0=h0$ so that $h^{-1}g$ fixes $0$, i.e. is linear. Then we may write $h^{-1}g=S\oplus R$ where $S$ and $R$ are rotations of $\Lambda$ and $\Lambda^{\perp}$ respectively. In fact we have

$$\begin{array}{ll} G & \cong (\Lambda,+)\rtimes\mathrm{SO}(\Lambda)\times\mathrm{SO}(\Lambda^{\perp}) \\ & \cong \mathbb{R}^r\rtimes\mathrm{SO}(r)\times\mathrm{SO}(n-r). \end{array} $$