For two non-square perpendicular matrices $X$ and $Y$ (i.e., $X^T Y = 0$), what happens when we bring a random matrix $A \sim \mathcal{N}(0,\sigma^2)$ (with i.i.d. elements) to the inner product, as follows?
$X^T (A^T A) Y = \text{?}$
Can we say anything about this multiplication?
Two obvious observations:
(a) $$<X^T A^T A Y> = X^T <A^T A> Y = X^T I Y = 0$$ where $<\cdots>$ means the expectation.
(b)
$$A^T A$$ is a matrix composed of products of i.i.d Gaussian variables, so the matrix $$M = X^T A^T A Y$$is a matrix composed of linear combinations of these products.
As for deeper results, I think that if you consider the generating function $$<e^{\lambda M}>$$ you can get quite a lot of information. This computation seems doable because the distribution is a gaussian and the exponential is quadratic in these variables and the integral will converge, at least if $\lambda<0$. that is: $$<e^{\lambda M}>=\int dA_{11}\cdots\int dA_{nn}e^{\lambda X^T A^T A Y-{1\over 2\sigma^2}\sum_{ab}A^2_{ab}}$$ At this point it becomes an exercise in Gaussian integrals which I leave as an "excercise for the reader". Whether that gives you enough information on the distribution is up to you, but in principle, it seems that generating functions are the way to proceed.