I know that the commutator is defined as $[A,B]=AB-BA$, but if $A$ and $B$ are operators, how do I multiply operators? For example, if I have
$$A=-4 \partial_x^3 + 6u(x,t) \partial_x + 3u_x$$
and
$$ B=-\partial_x^2 + u(x,t),$$
how do I obtain the commutator?
On
To calculate the commutator, it's best to do it by acting on a (sufficiently smooth) function. We interpret $AB$ and $BA$ as composition of operators. So, we'd want to calculate $$[A,B]f=A(Bf)-B(Af).$$ In this case, we find that \begin{align*} B(Af)&=(-\partial_x^2+u)(-4\partial_x^3 f+6u\partial_x f+3\partial_x uf)\\ &=4\partial_x^5 f-6(u\partial_x^3 f +2\partial_x u\partial_x^2 f+\partial_x^2 u\partial_x f)\\ &-3(\partial_x u\partial_x^2 f+2\partial_x^2 u\partial_x f+\partial_x^3 u f)\\ &-4u\partial_x^3 f+6u^2\partial_x f+3u\partial_x u f, \end{align*} for the second piece. I'll let you compute the first term, which is longer.
Take a function $f$, then define $[A,\,B]$ as the operator satisfying $[A,\,B]f=A(Bf)-B(Af)$. For example, if $g$ is a function and $I$ is the identity operator,$$[\partial_x,\,gI]f=\partial_x(gf)-g\partial_x f=(\partial_x g)f\implies[\partial_x,\,gI]=(\partial_x g)\cdot I.$$We usually drop the explicit $I$s, viz. $[\partial_x,\,gI]=(\partial_x g)$.