Namely if $V$ is a $\mathbb{R}$-linear vector space, with Euclidean inner product $\langle \cdot, \cdot \rangle$, and $\rho:G\rightarrow GL(V)$ is a representation of a compact Lie group $G$ on $V$, then there exists a new scalar product such that $\rho$ is an orthogonal representation.
The new scalar product, which we will call $\eta$ is defined by: $$\eta(v,w)=\int_G\langle \rho(g)v,\rho(g)w\rangle \sigma$$ where $\sigma$ is a nowhere vanishing, right invariant top form on $G$. To me, the key part of the proof that I'm interested is showing that $\eta(v,v)$ is an inner product, which follows from the fact that $\langle \rho(g)v,\rho(g)w\rangle$ is positive definite, and from well known statement about integration of top forms on smooth manifolds.
I am curious if we can relax the conditions on $\langle \cdot, \cdot\rangle$, to be only symmetric, bilinear, and nondegenerate. Does defining the new scalar product in the same way yield a `pseudo orthogonal' representation of $G$ on $V$? Would the new scalar product have the same signature as $\langle\cdot,\cdot\rangle$? If the process prescribed above doesn't work in this, then is there anyway to alter it so that it does?
Consider the following example: $G={\mathbb Z}_2$ (the group of 2 elements), $V={\mathbb R}^2$ equipped with the bilinear form $B$ whose quadratic form is $x^2-y^2$. Suppose that $G$ acts on $V$ so that its generator swaps the coordinate lines. I will leave you to check that the bilinear form $B'$ obtained from $B$ by averaging over $G$ is identically zero.