How can a representation of SO(3) be more than 3 dimensional?

Question

The group SO(3) can be represented (and defined) by 3-by-3 matrices as most of us have seen many times before.

But in "Group Theory in a Nutshell for Physicists", we "construct" an 8-dimensional representation by stacking two one dimensional and two 3-dimensional representations "on top" of each other, and we get an 8-dimensional (obviously reducible) representation. But I cannot comprehend how an 8-dimensional representation would even make sense when talking about rotations in 3 dimensions. The original 3-dimensional representation would work on a 3-dimensional vector but how would an 8-dimensional matrix be able to perform any meaningful operation on such a vector? Am I missing the point?

Answer 1

Yes, you are :-). You're looking at a very specific and rather anomalous case where a group happens to be defined in terms of linear maps on a vector space – that's not the best way to get to grips with the concept of a linear representation, since it invites you to confuse the defining representation with representations in general.

Better to start with a different example. The symmetric group $S_n$ can act linearly on all sorts of vector spaces. For instance, on the space of real-valued functions on $n$ elements, yielding an $n$-dimensional representation (which decomposes into a $1$-dimensional and an $(n-1)$-dimensional irreducible representation). Or on the space of real-valued functions on ordered pairs of $n$ elements, yielding an $n^2$-dimensional representation (which contains a copy of the $n$-dimensional representation in the previous example).

So a representation and its possible dimensions have nothing to do with any dimensions that may occur in the definition of the group, as most groups (like $S_n$) have nothing to do with vector spaces.

In the case of $SO(3)$, consider how the group acts on $3\times3$ matrices, where the action of an element $O\in SO(3)$ on a matrix $A$ is defined by $O^\top AO$. This is a linear action on the $9$-dimensional space of $3\times3$ matrices, and thus a $9$-dimensional representation of $SO(3)$. Multiples of the identity transform among themselves and thus form a $1$-dimensional subrepresentation. Antisymmetric matrices also transform among themselves and thus form a $3$-dimensional subrepresentation. Traceless symmetric matrices also transform among themselves and thus form a $5$-dimensional subrepresentation.

This $9$-dimensional representation happens to have a clear physical interpretation in $3$-dimensional space. For instance, the moment of inertia of a body is a symmetric $3\times3$ matrix, which you can decompose into a scalar part and a traceless part. When you rotate the system, these transform according to the above $1$-dimensional and $5$-dimensional subrepresentations, respectively. But representations don't need to have such interpretations; all that's required for a representation is the abstract property that the multiplication among a set of linear maps on a vector space corresponds to the group multiplication.

Answer 2

This is too simple to be a ‘real’ answer, but I feel like it might nonetheless be worth just having it written somewhere: the best way to think about representations, especially high-dimensional representations, is in terms of linear actions; it is entirely possible that a group can act on one vector space in one way, and on another vector space in an entirely different way, and this should not surprise us any more than that one can interpret $4$ as the area of a square when using $\sqrt4$ to compute the sidelength, or as arclength around a circle of radius $1$ when computing $\sin(4)$. This point of view @joriki has explained very well (and in much better language than my overly prosaic analogy).

However, it is also possible to think about representations in terms of matrices, and it appears that you feel most comfortable at this point doing so. In this connection, you can literally make 8-square matrices out of 3-square ones by stacking them: $$ \begin{pmatrix} a & b & c \\ d & e & f \\ g & h & i \end{pmatrix} \mapsto \begin{pmatrix} 1 \\ & 1 \\ && a & b & c \\ && d & e & f \\ && g & h & i \\ &&&&& a & b & c \\ &&&&& d & e & f \\ &&&&& g & h & i \end{pmatrix}, $$ or, if you really prefer all the $0$s,

$$ \begin{pmatrix} a & b & c \\ d & e & f \\ g & h & i \end{pmatrix} \mapsto \begin{pmatrix} 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & a & b & c & 0 & 0 & 0 \\ 0 & 0 & d & e & f & 0 & 0 & 0 \\ 0 & 0 & g & h & i & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & a & b & c \\ 0 & 0 & 0 & 0 & 0 & d & e & f \\ 0 & 0 & 0 & 0 & 0 & g & h & i \end{pmatrix}. $$ As @KyleMiller said, the resulting 8-square matrix is often referred to as the direct sum of the obvious two 1-square matrices and two 3-square matrices, and denoted, to save space, by $$ \begin{pmatrix} 1 \end{pmatrix} \oplus \begin{pmatrix} 1 \end{pmatrix} \oplus \begin{pmatrix} a & b & c \\ d & e & f \\ g & h & i \end{pmatrix} \oplus \begin{pmatrix} a & b & c \\ d & e & f \\ g & h & i \end{pmatrix}. $$ The similarity to the direct-sum terminology and notation for vector spaces is not accidental; the action of this matrix on the vector space $\mathbb R^8$ comes from viewing that space as $\mathbb R^1 \oplus \mathbb R^1 \oplus \mathbb R^3 \oplus \mathbb R^3$, and acting as indicated on each summand.

I hope it is clear both that we have produced an 8-square matrix, and how we have done so. Now, of course, the question is “why would we do such a thing?”—and for that, again, it is probably best to defer to @joriki's explanation.

On further reflection, one more thing: I think that also the terminology of ‘representation’ may be misleading here. It is natural to think that a representation of a group is supposed in some way to faithfully represent the elements of that group, but it need not do so—in fact, we have the terminology ‘faithful representation’ precisely to describe the situation where it does—or, in addition to or instead of faithfully representing group elements, that it should in some way be an efficient or natural representation of those elements. This was indeed some of the motivation for the formation of the theory, but, as is usual as theories develop, by this point we have abstracted away from the idea of how good, bad, efficient, or faithful the representation is, and now demand of a representation only that it be a linear action of whatever sort on whatever space; or, if we think on the level of matrices, that it be a homomorphism to an appropriate matrix group, here $\operatorname{GL}(8, \mathbb R)$. Thus, to say that we have an 8-dimensional representation of $\operatorname{SO}(3, \mathbb R)$ is to say literally nothing more than that we have a homomorphism $\operatorname{SO}(3, \mathbb R) \to \operatorname{GL}(8, \mathbb R)$. I have written one such, but there are others; for example, we could send every orthogonal matrix to the identity 8-square matrix.