Invariant Subrepresentation of Induced Representation

810 Views Asked by At

In what follows we let $G$ be a compact Lie group and $H\lneqq G$ a closed subgroup. We denote by $\mu_{G},\mu_{H}$ the respective Haar measures.

Let $(\mathcal{H},\langle,\cdot\,,\cdot\rangle_{\mathcal{H}})$ be a Hilbert space. A unitary representation $\Phi$ of a group $G$ on $\mathcal{H}$ is a group homomorphsim $\Phi:G\to\mathfrak{U}(\mathcal{H})$, where $\mathfrak{U}(\mathcal{H})$ is the group of unitary operators on $\mathcal{H}$, such that for each $v\in\mathcal{H}$ the map $g\mapsto\Phi(g)v$ is continuous. In what follows, we will write $gv$ for $\Phi(g)v$, whenever $\mathcal{H}$ is a Hilbert space, $\Phi:G\to\mathfrak{U}(\mathcal{H})$ a unitary representation of $G$ over $\mathcal{H}$, $g\in G$ and $v\in\mathcal{H}$. I was trying to prove the following:

Question: Let $\mathcal{H}$ be a finite-dimensional, unitary representation of $H\lneqq G$, then $\operatorname{Ind}_{H}^{G}\mathcal{H}$ contains a non-trivial, finite-dimensional, unitary subrepresentation of $G$.

I will first spend a few lines on defining all the symbols used above -- just to make sure we talk about the same objects and for you to find out whether I have in fact misunderstood the basics --, then I want to explain the context in which the question popped up and finally I want to outline what I intended to do.

Definition: Let $\mathcal{H}$ be a unitary representation of $G$. The left-regular representation of $G$ (over $\mathcal{H}$) is the unitary representation $G\to L^{2}(G,\mathcal{H})$ defined by: $$(gf)(x):=f(g^{-1}x)\quad\forall g,x\in G,\forall f\in\mathscr{C}(G,\mathcal{H})$$

Definition: Let $H\lneqq G$ be a closed subgroup, $\mathcal{H}$ a unitary representation of $H$. Set $$\mathcal{H}_{0}:=\{f\in\mathscr{C}(G,\mathcal{H});f(gh)=h^{-1}f(g)\forall g\in G,\forall h\in H\}$$ The induced representation $\operatorname{Ind}_{H}^{G}\mathcal{H}$ of $G$ is the closure of $\mathcal{H}_{0}\subseteq L^{2}(G,V)$ as a subrepresentation of the left-regular representation over $\mathcal{H}$.

Definition: Given a representation $\mathcal{H}$ of $G$ and $H\lneqq G$ a closed subgroup, we denote by $\operatorname{Res}_{G}^{H}W$ the representation of $H$ on $W$ obtained by restricting the map $G\to\mathfrak{U}(\mathcal{H})$ to $H$.

The question popped up as such a finite-dimensional, non-trivial unitary subrepresentation of the induced representation is chosen in an article. In particular, at the beginning of the proof of lemma 2.3 in Jinpeng An, Jiu-Kang Yu & Jun Yu: \emph{On the Dimension Datum of a Subgroup and its Application to Isospectral Manifolds}, Journal of Differential Geometry, \textbf{94} (2013), pp. 55-85 a non-trivial, finite-dimensional subepresentation of $\operatorname{Ind}_{H}^{H'}\mathbf{1}$ with $H\lneqq H'\leq G$ closed subgroups and $\mathbf{1}$ the trivial representation of $H$ is selected. I guessed that we could do this for any given finite-dimensional representation.

1st approach: I intended to use the Frobenius reciprocity together with the Peter-Weyl decomposition of $\operatorname{Ind}_{H}^{G}\mathcal{H}$, i.e. we note that for any representation $W$ of $G$ holds: $$\operatorname{Hom}_{H}(\operatorname{Res}_{G}^{H}W,\mathcal{H})\cong \operatorname{Hom}_{G}(W,\operatorname{Ind}_{H}^{G}\mathcal{H})$$ Now assume otherwise, i.e. all the finite dimensional subrepresentations of $\operatorname{Ind}_{H}^{G}\mathcal{H}$ are trivial, then application of Peter-Weyl implies that for all irreducible, finite-dimensional representations $W$ of $G$ holds: $$\operatorname{Hom}_{H}(\operatorname{Res}_{G}^{H}W,\mathcal{H})\cong\begin{cases} \mathbb{C}^{n} & \text{if }W\cong{\mathbf{1}_{G}}\\ \{0\} & \text{else} \end{cases}$$ where $n$ is the multiplicity of the trivial representation $\mathbf{1}_{G}$ in $\operatorname{Ind}_{H}^{G}\mathcal{H}$ (possibly infinity). I do not see how this can be pushed further to end up with a contradiction. All I get is that the components of $\mathcal{H}$ are all trivial which does not yield a contradiction by itself.

2nd approach: I intended to construct an element which spans a finite dimensional subspace which is not $G$-invariant. The idea was to use normality of Lie groups and compactness of $H$: we can construct a function $\phi:G\to\mathbb{C}$ such that $\phi\big|_{H}\equiv 1$ and $\phi\big|_{gH}\equiv 0$ for some $g\in G\setminus H$ and $\phi(G)\subseteq [0,1]$. Let $v\in\mathcal{H}\setminus\{0\}$, then the map $f:\hat{g}\mapsto \phi(\hat{g})v$ is continuous non-zero, hence for each $\hat{g}\in G$ we obtain a well-defined vector $\operatorname{Ind}f(\hat{g})$ by: $$\langle\operatorname{Ind}f(\hat{g}),w\rangle_{\mathcal{H}}:=\int_{H}\langle hf(\hat{g}h),w\rangle\operatorname{d}\mu_{H}(h)\quad \forall w\in\mathcal{H}$$ Using uniform continuity of $f$ it is not hard to deduce (uniform) continuity of $\operatorname{Ind} f$ and one easily sees $\operatorname{Ind} f\in\mathcal{H}_{0}$ thanks to the many invariance properties of $\mu_{H}$. The idea now was that clearly $\operatorname{Ind} f(g)=0$ and: $$\langle\operatorname{Ind} f(1),w\rangle_{\mathcal{H}}=\int_{H}\langle hv,w\rangle_{\mathcal{H}}\operatorname{d}\mu_{H}(h)\quad\forall w\in\mathcal{H}$$ So the question reduces to:

Is there a way to guarantee that for some $v\in\mathcal{H}$ the average $\int_{H}hv\operatorname{d}\mu_{H}$ defined by: $$\langle\int_{H}hv\operatorname{d}\mu_{H},w\rangle_{\mathcal{H}}=\int_{H}\langle hv,w\rangle_{\mathcal{H}}\operatorname{d}\mu_{H}(h)$$ is non-trivial?

Once the question is answered with yes, I can approximate the vector by matrix coefficients for irreducible representations of $G$ to obtain a non-zero vector in a finite-dimensional, non-trivial subrepresentation with non-trivial orbit under $G$. Note that with respect to the context the question popped up in, this construction indeed answers the question: if we induce the trivial representation, we integrate over $\lVert v\rVert_{\mathcal{H}}^{2}$, which is non-zero for $v\neq 0$.

1

There are 1 best solutions below

5
On BEST ANSWER

Edit: I see now that you want to rule out that $Ind_H^G \sigma$ is a multiple of the trivial representation. If $\sigma$ is nontrivial, then you are of course done, because $Res_H Ind_H^G \sigma$ contains $\sigma$. If $\sigma$ is trivial, and $H$ is normal you have that $Ind_H^G 1$ is the regular representation of $H\backslash G$, and if $H$ is a proper subgroup, there is at least one non-trivial representation of $H\backslash G$, conseqently in $Ind_H^G 1$.


The following addresses the discreteness of the induced representation, which I thought was the question at first reading.

The quotient of $H\backslash G$ is compact and $\sigma$ is unitary and finite-dimensional, hence the induced representation $(\pi, V_\pi)= Ind_H^G \sigma$ decomposes into a Hilbert direct sum of irreducible representations of $G$ with finite multiplicity.

One way how to prove that an induced representation decomposes discretely in the above sense is to show that $$ \pi (\phi): v \mapsto \int\limits_{G} \phi(g) \pi(g) v d g$$ is a compact operator for $\phi \in C_c^\infty(G)$. In fact, you obtain a Hilbert-Schmidt operator: Consider $V_\pi$ as the space of functions $f:G \rightarrow V_\sigma$ with $f(hg) = \sigma(h) f(g)$ and $\pi(g) f(x) = f(xg)$, then $$ \pi(\phi) : v \mapsto \int\limits_{G} \phi(x^{-1}g) f(g) d g,$$ so $\pi(\phi)$ is a kernel transformation with kernel $k(x,g) = \phi(x^{-1}g)$. The kernel is continuous, hence lies in $L^2(G \times G)$, which implies Hilbert Schmidt. Details can be found in "Principles of Harmonic Analysis" By Deitmar-Echterhoff (Chapter 9).

The operator is actually trace class, but this requires the Dixmier-Malliavin theorem, which lies deeper.

This strategy works, whenever $H$ is cocompact in $G$. It fails for example for $SL_n(\mathbb{Z}) \subset SL_n(\mathbb{R})$, which is a cofinite embedding for $n \geq 2$.