Find particular solution of $$u''+2u'+u=9x^2e^{-x}.$$ So I've already worked out that the particular solution is $Ae^{-x}+Bxe^{-x}$ so then I tried the particular solution $u_p=Cx^2e^{-x}$ but it didn't work.
Anybody know why it didn't work and what I should try instead?
Since the polynomial on the right-hand side is of second degree and $-1$ is a root of the characteristic polynomial with multiplicity $2$, you should try the following more general form $$u_p(x)=x^2(Ax^2+Bx+C)e^{-x}.$$ More generally if the right-hand side is $e^{rx}P(x)$ where $P$ is a polynomial of degree $d$ and $r$ is a root of the characteristic polynomial with multiplicity $m$ then use the form $$u_p(x)=x^mQ(x)e^{rx}$$ where $Q$ is a generic polynomial of degree $d$.