$O(2)$ subgroup representation of $O(3)$

Question

My objective is to:

Argue that the defining representation of $O(3)$ is irreducible and becomes reducible when restricting to an $O(2)$ subgroup.

The defining rep of $O(3)$ is:

$ \begin{pmatrix} a & b & c \\ -b & d & e \\ -c & -e & f \end{pmatrix}$

which I can argue is an irrep because (geometrically) it has no invariant subspaces, and (algebraically) it is an irrep according to Schur's lemma.

I'm completely confused how to make a $2-\dim O(2)$ rep version of this :(

Answer 1

Any $2\times 2$ matrix can be turned into a $3\times 3$ one by making it block-diagonal, with the original $2\times 2$ matrix first and then a $1\times 1$ block with the value $1$ next. This is a very general way that classical matrix groups embed into classical matrix groups of higher dimensions.

Here, geometrically speaking, $O(2)$ acts on the first two coordinates of $\mathbb{R}^3$, so by rotations and reflections within the $xy$-plane. Can you find $1$D and $2$D invariant subspaces?