Suppose $\delta_1,\delta_2$ are two inequivalent representations of a compact Lie group. Let $dy$ be the normalised Haar measure and define convolution for functions $f,g:G\rightarrow \mathbb{C}$
$$f*g(x)=\int_Gf(xy^{-1})g(y)dy$$
Now denote by $C(G)_{\delta_i}$ for $i=1,2$ the linear span of matrix coefficients. I want to show that for $f\in C(G)_{\delta_1}$ and $g\in C(G)_{\delta_2}$ we have $f*g=0$. This is how I proceeded:
We know that $\langle f,g\rangle=\int_Gf(y)\overline{g(y)}dy=0$ when $f\in C(G)_{\delta_1}$ and $g\in C(G)_{\delta_2}$ and $\delta_1$ and $\delta_2$ are inequivalent. So I want to use this somehow:
$$f*g(x)=\int_Gf(xy^{-1})g(y)dy=\int_Gf(y^{-1})g(x^{-1}y)dy=\int_G\overline{f(y)}g(x^{-1}y)dy$$
Now this looks almost like $\langle f,g\rangle$. So how should I proceed from here? Thanks.
First I think you want to assume both reps are irreducible, right?
Now if $g(y)=\eta(\delta(y)v)$ is a matrix coefficient ($\eta\in V_\delta^\ast, v\in V_\delta$), then $y\mapsto g(x^{-1}y)$ is another matrix coefficient: $$g(x^{-1}y)= \eta(\delta(x^{-1})\delta(y)v)= (\eta\circ \delta(x^{-1}))(\delta(y)v)$$