The following is an exercise from Representation Theory: A First Course, by Fulton and Harris. Yes, I am assigned this problem as homework. I ask only for help understanding what the claim actually is.
Exercise 2.35: Show that, if the irreducible representations of $G$ are represented by unitary matrices, the matrix entries of these representations form an orthogonal basis for the space of all functions on $G$ ,with inner product given by $$ (\alpha,\beta) = {\frac{1}{|G|}} \sum_{g \in G} \overline{\alpha(g)} \beta(g) $$ Based on context, we should assume $G$ is finite and we're talking about representations over $\mathbb{C}$. My question is, what does this claim mean?
What does it mean that "the irreducible representations of $G$ are represented by unitary matrices"? Does this mean that for each irreducible representation $\rho:G \to \operatorname{GL}_n(\mathbb{C})$, the image is contained in the subgroup $U(n)$ of unitary matrices?
The statement seems to imply that the representations themselves are somehow in correspondence with unitary matrices, not the actions of group elements. In particular, the reference to "matrix entries of these representations" supports that kind of interpretation. I can't make sense of this phrase either.
I assume the phrase "functions on $G$" refers to functions $G \to \mathbb{C}$, but I could be wrong about that.
Any help is much appreciated. Thanks!
Before that exercise, you have exercise 1.14, which states:
So, if $G$ acts on $V$, you can assume that $V$ is endowed with a Hermitian inner product $\langle\cdot,\cdot\rangle$ preserved by $G$, and this means that$$(\forall g\in G)(\forall v,w\in V):\langle gv,gw\rangle=\langle v,w\rangle.$$So, the action of each $g\in G$ is a unitary transformation and therefore, if you fix an orthonormal basis of $V$, you can see the action of $g$ as an unitary matrix (the matrix of the action of $g$ with respect to $b$).
And, yes, here the functions on $G$ are functions from $G$ into $\mathbb C$.
Example: Consider the action $\rho$ of $\mathbb{Z}_3$ on $\mathbb{C}^2$ defined by$$\rho(n)=\begin{pmatrix}-\frac12&-\frac{\sqrt3}2\\\frac{\sqrt3}2&-\frac12\end{pmatrix}^n$$($n\in\{0,1,2\}$). These matrices are unitary. Let $\alpha\colon\mathbb{Z}_3\longrightarrow\mathbb C$ be the upper left entry of these matrices. Then $\alpha(0)=1$, and $\alpha(1)=\alpha(2)=-\frac12$. And if $\beta\colon\mathbb{Z}_3\longrightarrow\mathbb C$ is the upper right entry of these matrices, then $\beta(0)=0$, $\beta(1)=-\frac{\sqrt3}2$, and $\beta(2)=\frac{\sqrt3}2$.