I got a problem from the book An Introduction to Lie Groups and Lie Algebras written by Kirillov.
The exercise 5.1 says $$\begin{array}{l} \text { (1) Let } V \text { be a representation of } \mathfrak{g} \text { and } W \subset V \text { be a subrepresentation. Then } B_{V}=B_{W}+B_{V / W} \text { , } \\ \text { where } B_{V} \text { is defined by }(5.14) .\end{array}$$
and 5.14 says $$\text { Let } V \text { be a representation of g and define a bilinear form on g by } B_{V}(x, y)=\operatorname{tr}_{V}(\rho(x) \rho(y)).$$
I'm not sure what $B_{V/W}$ is. For $\rho_{V/W}(x)$, it should be defined as $\rho_{V/W}(x)(v+W)=\rho_V(x)(v)+W$. However, what should $\operatorname{tr}_{V/W}$ be? I can't find any information about this trace. Is there any common definition for it? Thank you.
Your definition of $\rho_{V/W}$ is wrong. $\rho_{V/W}$ is the representation of $\mathfrak{g}$ on $V/W$ so it is $\rho_{V/W}(x)(v+W)=\rho_V(x)(v)+W$ for all $x\in\mathfrak{g}$ and all $v\in V$. You can take trace of linear maps $V/W\to V/W$ so that gives you $\operatorname{tr}_{V/W}(\rho_{V/W}(x)\rho_{V/W}(y))$ which of course you can drop some of the $V/W$ subscripts for brevity.