An example of Schur's lemma

Question

Reading Linear representations of finite groups by Serre, I need an example of the following:

Schur's lemma:

Let $\rho^1\colon G \rightarrow GL(V_1)$ and $\rho^2\colon G \rightarrow GL(V_2)$ be two irreducible representations of a group $G$, and let $f$ be a linear mapping of $V_1$ into $V_2$ such that $\rho_s^2 \circ f = f \circ \rho_s^1$ for all $s \in G$. Then:

(i) If $\rho^1$ and $\rho^2$ are not isomorphic, we have $f=0$

(ii) If $V_1 = V_2$ and $\rho^1 = \rho^2$, $f$ is scalar multiple of the identity.

Can someone form a concrete example so I can see what's going on?

Answer 1

If you translate Schur's Lemma into the language of representations of finite groups, you get the following.

---

Let $G$ be a finite group, $\mathsf{k}$ some field, and $\rho_i : G \to \operatorname{GL}\left(V_i\right)$ some irreducible representations of $G$ over $\mathsf{k}$. Then

1) If $\rho_1$ is not equivalent to $\rho_2$, then there are no non-trivial $\mathsf{k}$-linear maps $T : V_1 \to V_2$ intertwining $\rho_1, \rho_2$.

2) If $\mathsf{k}$ is algebraically closed, and $\rho_1, \rho_2$ are equivalent, then every non-trivial $\mathsf{k}$-linear map $T : V_1 \to V_2$ intertwining $\rho_1, \rho_2$ is an isomorphism.

---

By a linear map $T : V_1 \to V_2$ intertwining $\rho_1, \rho_2$, what I mean is that $T\left( \rho_1(g) v \right) = \rho_2(g) T(v)$ for all $g \in G$, $v \in V$, and two representations $\rho_1, \rho_2$ are called equivalent if there is a linear isomorphism intertwining them.

---

Schur's lemma is a direct result of the following observation

Let $G$ be a finite group, $\mathsf{k}$ some field, and $\rho_i : G \to \operatorname{GL}\left(V_i\right)$ some representations of $G$ over $\mathsf{k}$, and suppose that $T : V_1 \to V_2$ is a $\mathsf{k}$-linear map intertwining $\rho_1$, $\rho_2$. Then

1) $(\rho_1, \operatorname{Ker}{T})$ is a subrepresentation of $(\rho_1, V_1)$.

2) $(\rho_2, \operatorname{Img}{T})$ is a subrepresentation of $(\rho_2, V_2)$.

---

Now, if $(\rho_i, V_i)$ are irreducible, and $T : V_1 \to V_2$ is a $\mathsf{k}$-linear map intertwining $\rho_1$, $\rho_2$, then $\ker{T}$ is either $\left\{0\right\}$, or $V_1$. In the first case, $T$ is injective, whilst in the second case $T$ is the trivial map. Now suppose we have the first case, then since $V_2$ is irreducible, we have $\operatorname{Img}T$ is either $\left\{0\right\}$ or $V_2$. So our options are

1. $V_1 = \left\{0\right\}$, in which case $T = 0$,
2. $V_1 \neq \left\{0\right\}$ and $T=0$,
3. $V_1 \neq \left\{0\right\}$ and $T \neq 0$, in which case $T$ is injective and surjective, and so $T$ is a isomorphism intertwining $V_1, V_2$

Now suppose that we are in the third case, and $V_1 = V_2$, and that $\mathsf{k}$ is algebraically closed, and suppose that $T : V_1 \to V_1$ is a non-trivial intertwining map. Then by the above $T$ is an isomorphism, and since $\mathsf{k}$ is algebraically closed, $T$ has an eigenvector $0 \neq v_1 \in V_1$, with eigenvalue $\lambda$ say. Then it is easy to see that the $\lambda$-eigenspace of $T$ is a non-zero subrepresentation of $V_1$, and since $V_1$ is irreducible we have that $V_1$ is the $\lambda$-eigenspace of $T$, and so $T(v) = \lambda v$ for every $v \in V_1$, and so $T = \lambda \operatorname{Id}_{V_1}$, and moreover $\lambda \neq 0$. It follows that every non-trivial intertwining map $T : V_1 \to V_1$ is a scalar multiple of the identity.

---

Now, for a concrete example. Consider the representation of $S_3$ defined by $$ \rho_V : S_3 \to \operatorname{GL}(\mathbb{C}^{3}) \ : \ \rho_V(\sigma)(ae_1 + be_2 + ce_3) = ae_{\sigma(1)} + be_{\sigma(2)} + ce_{\sigma(3)}. $$

Then $\rho_V$ is not irreducible, because the two subspaces

$$ U = \left\{ae_1 + be_2 + ce_3 \mid a,b,c \in \mathbb{C} : a + b + c = 0 \right\}, \\ W= \left\{a(e_1 + e_2 + ce_3) \mid a \in \mathbb{C} \right\} $$

are both subrepresentations. Moreover, it is clear to see that $U \cap W = \left\{ 0 \right\}$, and so for dimension reasons we have $V = \mathbb{C}^{3} = U \oplus W$. Now, $W$ is one-dimensional and so must be irreducible, and take a moment to convince yourself that $U$ is irreducible too.

Now consider the representation $\rho_t : S_3 \to \mathbb{C}^{\times}$ such that $\rho_t(\sigma) = t$ for every $\sigma$, where $t \neq 0$. Suppose that $T : \mathbb{C}^{\times} \to U$ is an intertwining map, and suppose $T(1) = ae_1 + be_2 + ce_3$. Then since $T$ intertwines, we have $$ t(ae_1 + be_2 + ce_3) = tT(1) = T(t) = T(\rho_t(\sigma)1) = \rho_V(\sigma)(T(1)) = \rho_V(\sigma)(ae_1 + be_2 + ce_3), $$

and so

$$ tae_1 + tbe_2 + tce_3 = ae_{\sigma(1)} + be_{\sigma(2)} + ce_{\sigma(3)}, $$

for every $\sigma \in S_3$. Taking $\sigma = \operatorname{Id}$ and $\sigma = (12)$ gives $ta = a = b$, and so $a = b$. Similarly you can show $a = c$ and $c=b$, but $a + b + c =0$, and so we get $a = b = c =0$. So every linear map intertwining $\rho_t$, and $\rho_V$ is zero, for whatever choice of $\rho_t$ we choose.

Now, suppose instead that $T: \mathbb{C}^{\times} \to W$ is an intertwining map, suppose that $T(1) = a(e_1 + e_2 + e_3)$. Then it is clear that $T(x) = xa(e_1 + e_2 + e_3)$, and so $T$ is a scalar multiple of the map $I : \mathbb{C}^{\times} \to W$ sending $1$ to $e_1 + e_2 + e_3$. This works regardless of the choice of $t$ (so long as $t \neq 0$ because $(\rho_t, \mathbb{C}^{\times})$ and $(\rho_V, W)$ are equivalent representations.