How would I go about solving this question? I've treid letting $u = a^2-x^2$
$$\begin{align}I&=\int_0^a x\sqrt{a^2-x^2}\,\mathrm dx\\&=-\dfrac12\int_0^a\sqrt{a^2-x^2}\,\mathrm d(a^2-x^2)\\&=-\dfrac12\int_{x=0}^{x=a}\sqrt u\,\mathrm du\\&=-\dfrac12\cdot \dfrac23u^{\frac32}\bigg|_{x=0}^{x=a}\\&=-\dfrac13(a^2-x^2)^{\frac32}\bigg|_0^a\\&=0+\dfrac13(a^2)^{\frac32}\\&=\dfrac{a^3}3\end{align}$$
It's pretty easy, and the u-sub is correct, so you're on the right track!
$$\int_0^a x\sqrt{a^2-x^2}\ dx$$
Let $u = a^2-x^2$. Then $du = -2x$, and when we sub this in and change the bounds $(u(0)=a^2,\ u(a)=0)$, we get $$\int_{a^2}^0 -\frac12\sqrt u\ du$$ $$=-\frac13u^{3/2}|_{a^2}^0 = \frac{a^3}3$$
So what did you need help with, anyway?