I have to solve the following problem:
Obtain a matrix whose entries are the expectation values of a randomly chosen matrix belonging to $SO(n)$ for $n \geq 2$.
I could set up the expression that would give me the result, but I have trouble evaluating it. I feel lost.
Let $\mu$ be the Haar measure of $SO(n)$ and $e_i$ be the $i$th unit vector in $\mathbb{R}^n$. Then, we want the matrix
$$\mathbf{E} = (E)_{ij} = \frac{1}{\mu(SO(n))} \int_{SO(n)} (e_i \cdot U e_j) \mathrm{d}\mu(U)$$
where $U$ is an arbitrary randomly chosen matrix in $SO(n)$. Furthermore, we know also that $\mu(SO(n))<\infty$ since $SO(n)$ is compact.
After here, I am lost. Any help is appreciated!
Let $G$ be any compact, locally compact matrix group; let $\mu$ be its normalized Haar measure, i.e. $\mu(G)=1$. Let $E=\int\limits_Gg\, d\mu(g)$ be the expectation we are interested in. The key point is that $E$ is invariant under the action of $G$. Consider $g_1 \in G$. Then, $g_1 \cdot E = \int_G g_1 g \, d\mu(g) = \int_G h \,d\mu(g_1^{-1}h) = \int_Gh \, d\mu(h) = E$ (we made a substitution $h=g_1g$ and used invariance of Haar measure).
Now consider $SO(n)$. First, if $n=1$, this group is trivial, so the expectation is just the identity matrix. Now, suppose that $n \geq 2$. Let $g$ be a $180^\circ$ rotation in $ij$ coordinate plane (this is where we need $n \geq 2$). Multiplying a matrix $E$ by $g$ on the left has the effect of multiplying by $-1$ the $i$-th and $j$-th rows of $E$. Since $E$ is invariant, $gE=E$, meaning that any row in $E$ doesn't change if we multiply it by $-1$, meaning that any row of $E$ consists of zeros. Thus, $E=0$.
NB: I once answered a somewhat related question about finite groups; you may be interested in it.