How to generate orthogonal matrices?

Question

An orthogonal matrix has $\frac{n(n-1)}{2}$ degrees of freedom.

Does there exist a function that maps $\frac{n(n-1)}{2}$ variables to the $n^2$ parameter orthogonal matrix?

And can such a function be differentiable?

The actual objective behind this problem is to feed some input through a random orthogonal matrix and obtain some result which is then fed through a loss function and the gradients are used to optimize the orthogonal matrix. Currently the approach I'm using is imposing a $L_1(W W^T - I)$ penalty to approximate this - but note how this doesn't prevent $W$ from being non-orthogonal just nudges it in that direction.

Answer 1

An $n \times n$ orthogonal matrix is a matrix whose rows (or columns, if you prefer) form an orthonormal basis of $\mathbb R^n$. You might start with $n$ random vectors and use Gram-Schmidt to orthonormalize them. If you run into numerical difficulty (because one of your vectors is too close to the linear span of the previous ones), choose a new random vector to replace it.

Answer 2

We consider the Cayley transform: $f:A\in SK_n\rightarrow (I_n-A)(I_n+A)^{-1}\in \Omega_n$ where $SK_n$ is the set of $(n\times n)$ skew-symmetric matrices and $\Omega_n=\{U\in O^+(n)\;;\;-1\notin spectrum(U)\}$.

$f$ is an algebraic diffeomorphism. It is not onto on $O^+(n)$ but it does not matter if we randomly work on $O^+(n)$. Indeed, a randomly chosen $A\in O^+(n)$ has with probability $1$:

no real eigenvalues when $n$ is even; exactly $1$ real eigenvalue (equal to $1$) when $n$ is odd.

Indeed, consider the function $A\in SK_n\rightarrow e^A\in O^+(n)$ and the fact that a random $A$ is invertible when $n$ is even and has exactly $1$ zero eigenvalue when $n$ is odd.

Then, to make a random choice on $O(n)$, you can proceed as follows:

Randomly choose $A\in SK_n$, calculate $f(A)=[C_1,\cdots,C_n]$; select at random $\pm 1$. If $+$, then output $f(A)$, else output $[-C_1,\cdots,C_n]$.

Of course, the probability law on $O(n)$ will depend on the probability law that you choose on $SK_n$.

Answer 3

I've found a way to construct all orthogonal matrices from $\frac{n(n-1)} 2$ variables, and hence all matrices by SVD, as below,

1. first choose a unit vector from spherical coordinates, with $(n-1)$ variables, $$v_n=[\cos(x_1), \sin(x_1)\cos(x_2), \sin(x_1)\sin(x_2)\cos(x_3), ..., \sin(x_1)..\sin(x_{n-1})]$$
2. construct n-1 orthogonal vectors from $v_n$, based on Istvan Mayer's Simple theorems, proofs, and derivations in quantum chemistry Appendix XI:
   first column is $v_1$(sorry I'm not allowed to embed a pic here.)
   as a matrix $H_n[n,n-1]$;
3. construct another orthogonal matrix $A_{n-1}[n-1, n-1]$, same way as above steps;
4. final result recursively: $$A_n[n, n]=[v_n, H_n*[v_{n-1}, H_{n-1}*[..., H_3*A_2]..]$$

this maps $\frac{n(n-1)} 2$ variables to all orthogonal matrices.

By SVD, we may build a weight matrix with spectral norm strictly controlled, this should be better than regulation/normalization methods, I'm trying this..