I was wondering whether it was possible to derive the kth moment of an exponential random variable $X \sim \text{EXP}(\theta)$ without using the MGF and then finding the kth derivative.
I came up with this derivation:
Let $X \sim \operatorname{EXP}(\theta)$
Then the PDF of $X$ is:
$$f(x) = \begin{cases} \frac{1}{\theta} e^{\frac{-x}{\theta}} & x \gt 0 \\ 0 & \text{otherwise} \\ \end{cases}$$
The definition of the kth moment is $\operatorname{E}(X^k)$, so
$$\begin{align} \operatorname{E}(X^k) & = \int_{-\infty}^\infty x^k \; f(x) \; dx \\ & = \int_{0}^\infty x^k \; \frac{1}{\theta} e^{\frac{-x}{\theta}} \; dx \\ & = \frac{\theta^k}{\theta^k} \int_{0}^\infty x^k \; \frac{1}{\theta} e^{\frac{-x}{\theta}} \; dx\\ & = \theta^k \int_{0}^\infty x^k \; \frac{1}{\theta^{k+1}} e^{\frac{-x}{\theta}} \; dx\\ & = \frac{\Gamma(k+1)}{\Gamma(k+1)} \theta^k \int_{0}^\infty x^k \; \frac{1}{\theta^{k+1}} e^{\frac{-x}{\theta}} \; dx\\ & = \Gamma(k+1) \theta^k \int_{0}^\infty x^k \; \frac{1}{\Gamma(k+1) \theta^{k+1}} e^{\frac{-x}{\theta}} \; dx \quad (*) \\ \end{align}$$
Since $\int_{0}^\infty x^k \; \frac{1}{\Gamma(k+1) \theta^{k+1}} e^{\frac{-x}{\theta}} \; dx$ is the PDF of $X \sim \Gamma(k+1, \theta)$, it equals one, so:
$$ \begin{align} \operatorname{E}(X^k) = \cdots & = \Gamma(k+1) \theta^k \int_{0}^\infty x^k \; \frac{1}{\Gamma(k+1) \theta^{k+1}} e^{\frac{-x}{\theta}} \; dx \quad (*) \\ & = \Gamma (k+1) \theta^k \quad \square \end{align}$$
Is this derivation correct?
There are no mistakes in the derivation but that's all. Things can be done much easyer.
First let me note that it is common to write $\lambda e^{-\lambda x}$ for the PDF of exponential distribution on positive axis. Apparantly you go out from $\theta=\frac{1}{\lambda}$. There is nothing wrong with that, but it must be mentioned to avoid confusion.
Secondly you could save yourself a lot of work by stating that your $X$ has the same distribution as $\theta Y$ where $Y$ has exponential distribution with parameter $1$.
Then $\mathbb EX^k=\theta^k\mathbb EY^k$ so based on this observation it is enough to prove that $\mathbb EY^k=\Gamma(k+1)$. Parameters are annoying and if you do without then the probability on mistakes is much less.
You ended with stating that "$\int_0^{\infty}\cdots$ is the PDF of $X\sim\Gamma(k+1,\theta)$, so it equals $1$".
But it is a better idea to start with equivalent statement for $\theta=1$ that: $$\Gamma(k+1)=\int_0^{\infty}x^ke^{-x}dx$$
It tells you immediately that $\mathbb EY^k=\Gamma(k+1)$ and consequently that $\mathbb EX^k=\theta^k\Gamma(k+1)$.