I used the substitution $u=\cos x$ whereas my teacher used $u=\sin x$ and we get completely different answers of course, which do not seem to be equivalent. I don't understand how a single integral can have several solutions.
My answer: $-\frac{\cos^4x}{4}+\frac{2\cos^6x}{6}-\frac{\cos^8x}{8} + C$
My teacher's answer: $\frac{\sin^6x}{6}-\frac{\sin^8x}{8} + C$
Both of your solutions are missing the additive constant, so your solutions should be:
Your answer: $-\frac{\cos^4x}{4}+\frac{2\cos^6x}{6}-\frac{\cos^8x}{8} + C$
Your teachers answer: $\frac{\sin^6x}{6}-\frac{\sin^8x}{8} + C$
for each additional constant $C \in \Bbb R$
And you are both right :-)
Both of you found an antiderivative of $\sin^5(x)\cos^3(x)$ because of:
$$\frac{d}{dx}\left(-\frac{\cos^4x}{4}+\frac{2\cos^6x}{6}-\frac{\cos^8x}{8}\right) = \sin^5(x)\cos^3(x)$$
and
$$\frac{d}{dx}\left(\frac{\sin^6x}{6}-\frac{\sin^8x}{8}\right) = \sin^5(x)\cos^3(x)$$
Additionally you know now that both of your solutions only differs by a constant $C \in \Bbb R$.