I'm reading Representation Theory of Finite Groups, by Benjamin Steinberg.
He derives the following proposition(p.18) after a small discussion before that,
Proposition 3.1.19. If $\varphi:G\to GL(V)$ is a representation of degree $2$(i.e., $\dim V=2$), then $\varphi$ is irreducible if and only if there is no common eigenvector $v$ to all $\varphi_g$ with $g\in G$.
He writes the following right after
Notice that this trick of using eigenvectors only works for degree $2$ and degree $3$ representation(and the latter case requires finiteness of $G$).
I don't understand why one can use the above proposition for degree $3$ representation and why $G$ required to be finite in this case. Moreover, why in degree $2$ representation $G$ allowed to be infinite group?
If a 2D rep has a subrep, the subrep must be 1D, and so any vector in it is consequently an eigenvector for each group element.
If a 3D rep has a subrep, then either the subrep is 1D and the same logic applies, or it's 2D, in which case if $G$ is also finite then the usual trick shows it has a complementary subrep, which must be a 1D rep, and hence the same logic applies in that case too.
Recall the "usual trick" (also called the unitary trick, averaging, gets attributed to Weyl, etc.) is to take an inner product $\langle-,-\rangle$, average as $\langle x,y\rangle_G=|G|^{-1}\sum_{g\in G}\langle gx,gy\rangle$, which must also be an inner product (exercise), and then show the orthogonal complement of a subrep wrt $\langle-,-\rangle_G$ must also be a subrep. Sometimes this trick is done with projection operators instead of inner products.