I am working on a problem concerning a Lie group acting on a smooth manifold or more generally, groups acting on topological manifolds or topological spaces. I am wanting to become more familiar with this topic. I would like to know things like, are the orbits manifolds, is the quotient space a manifold, what are the dimensions, etc.
The particular problem I am working on is the action $O(n) \times \mathbb{R}^{n \times p} \rightarrow \mathbb{R}^{n \times p}$ given by $(U, X) \mapsto UX$.
Im not sure if it is possible, but the ultimate goal would be to find a set $T$ and a map $f: T \rightarrow \mathbb{R}^{n \times p}$ such that for each $t_1, t_2 \in T$, $t_1 \neq t_2$ implies that $\pi(f(t_1)) \neq \pi(f(t_2))$ and $\bigcup_{t \in T} \{\pi(f(t))\} = \mathbb{R}^{n \times p} / O(n)$ so that we can parametrize the quotient in way to pick out a single element of each orbit.
In the case $p=1$, it looks like the orbits are concentric $n-1$ dimensional spheres and the quotient $\mathbb{R}^n / O(n)$ is just $[0, \infty)$ and we could have $T = [0, \infty)$ and $f: T \rightarrow \mathbb{R}^n$ given by $f(t) = te_1$ where $e_1 = (1, 0, \ldots, 0)^T$
However, in the $p > 1$ case, I can't visualize the orbits as easily.
I was thinking that since $$ U \begin{pmatrix} X_1 & \ldots & X_p \end{pmatrix} = \begin{pmatrix} UX_1 & \ldots & UX_p \end{pmatrix} $$ maybe the orbits could be determined by the length of the columns $X_j$ and the angles between the columns which can be specified by less than or equal to $p + {p \choose 2} = {p+1 \choose 2}$ parameters. So if the quotient space is a manifold, then I was thinking it would have dimension at most ${p+1 \choose 2}$ but I feel like I just don't know enough yet to make progress.
Most of the stuff I saw in books like Lee's intro to smooth manifolds was in the context of a freely acting Lie group on a smooth manifold, but in this case, I don't think the action is free right? So maybe smooth manifolds isn't the right context for this problem, I'm not sure. Can anyone recommend some resources like books/videos to read/watch to start developing intuition for these types of things?
For any matrix $X\in M_{n\times p}\mathbb{R}$ there is an upper triangular matrix $Y$ with nonnegative diagonal entries such that $Y=UX$ for some $U\in O(n)$. Indeed, the diagonal entries of $Y$ will be the magnitudes of the column vectors of $X$. This is essentially the QR decomposition of $X$. You can construct $U$ by first picking an element of $O(n)$ to turn the first column into a vector with only its top entry nonzero, then pick an element of $O(n-1)$ acting on the next $n-1$ coordinates to turn the second column into a vector with at most its top two entries nonzero, and so on.
If we augment this procedure slightly to skip any columns of $X$ that are entirely zero, we get a unique "staggered" upper triangular matrix $Y$, by which I mean it is upper triangular after we delete any all-zero columns of it. The "staggered" upper triangular matrices work as a transversal for the orbits, therefore. The stabilizers depend on how far down the positive diagonal entries (in the zero-column-deleted matrix) go compared to the column height.
The "staggered" upper triangular matrix format with the most degrees of freedom (parameters defining it) is the one which has no zeroed columns at all. If $p\le n$ this is $\binom{p+1}{2}$ parameters and beyond that if $p>n$ it is $\binom{p+1}{2}+(p-n)n$ parameters (because after the diagonal reaches the bottom, the rest of the matrix to the right is arbitrary).