Consider a finite group $G$, a subgroup $H\leq G$, and a transversal $G/H = \{t_1H, t_2H,\ldots,t_rH\}$. Given three characters $\chi_1,\chi_2$ and $\chi_3$ of $G$, I'd like to compute:
$$ \sum_{i}^r \left[\left(\sum_{\sigma_1\in t_iH}\chi_{1}(\sigma_1)\right)\left(\sum_{\sigma_2\in t_iH}\chi_{2}(\sigma_2)\right)\left(\sum_{\sigma_3\in t_iH}\chi_{3}(\sigma_3)\right)\right] $$
Actually, I just need to know if it is zero or not.
In the case where $H=\{e\}$ (the trivial group) we get
$$ \sum_{\sigma\in G}\chi_1(\sigma)\chi_2(\sigma)\chi_3(\sigma), $$
and we can reinterpret this as the multiplicity of the trivial rep of $G$ in the (tensor) product rep $\chi_1\otimes \chi_2\otimes \chi_3$ of $G\times G\times G$ restricted to $G$ via $g \mapsto (g,g,g )$.
In the case where $H=G$ we get
$$ \left(\sum_{\sigma_1\in G}\chi_{1}(\sigma_1)\right)\left(\sum_{\sigma_1\in G}\chi_{2}(\sigma_2)\right)\left(\sum_{\sigma_3\in G}\chi_{3}(\sigma_3)\right), $$
and we can reinterpret this as the product of the multiplicities of the trivial rep of $G$ in each of $\chi_1$, $\chi_2$ and $\chi_3$. Hence if $\chi_1$, $\chi_2$ and $\chi_3$ are irreducible we get a non-zero answer iff each character is trivial.
In the intermediate cases with $H$ a proper subgroup, I was hoping it would also just be a matter of reinterpreting using inner products of induced characters $H\uparrow G$, but this doesn't seem to work out (unless I am missing something).
As per my question title you see it comes down to average values of the characters on each coset, but I can't see a nice way of understanding what this is telling me.
Failing a theoretical answer, I've started coding up in GAP to compute specific examples. Any suggestions on how this could be achieved efficiently would also be very much appreciated.
Thanks!
The product will be zero (for any choice of coset representative $t_{i}$ and for each coset) if any of $\chi_{1},\chi_{2},\chi_{3}$ contain the trivial character with multiplicity zero on restriction to $H.$ However, I do not think the converse ( for a particular choice of $t_{i}$) need be true. It suffices to note that if the complex (matrix) representation $\sigma$ of the finite group $H$ does not contain the trivial representation, then $\sum_{h \in H} (h \sigma) = 0.$ It suffices to check this for each irreducible constituent of $\sigma,$ by Maschke's Theorem. So it suffices to check the case that $\sigma$ is irreducible, but non-trivial. Then $\sum_{h \in H} h\sigma$ commutes with $x\sigma$ for each $x \in H.$ Hence it is a scalar matrix by Schur's Lemma. But it has trace zero by the orthogonality relations, so it is the zero matrix.
Note then, that for general $\sigma$ not contaig the trivial representation on restriction to $H,$ and any $t \in G,$ we have $\sum_{h \in H}(th\sigma) = (t\sigma)\sum_{h \in H}h\sigma = [0].$