I have managed to do most of the question, but can't get the very last part. We have $M_{2Np}(t)=M_N(2pt)$, but I don't seem to be getting a limiting gamma distribution.
For $Y$ I got the MGF as $\frac{p~\mathsf e^t}{(1-\mathsf e^t~(1-p))}$ (because Y has geometric distribution), and for $N$ I think it is a sum of $k$ independent random variables with the same distribution as $Y$, so the MGF should be $(M_Y(t))^k$ .
Would appreciate if someone could show me how to do the last part. Thanks!

Okay, you have: $$M_N(t)= \dfrac{p^k~\mathsf e^{kt}}{(1-(1-p)\mathsf e^{t})^k}\qquad\color{green}\checkmark$$
Then by substitution. $$\begin{align}M_{2Np}(t) =&~ M_N(2pt) \\[1ex]=&~ \dfrac{p^k~\mathsf e^{2kpt}}{(1-(1-p)\mathsf e^{2pt})^k}\end{align}$$
Now show that $$M_X(t) = \lim\limits_{p\downarrow 0} M_{2Np}(t)$$
Where $M_X(t)$ is the MGF of $X$, the given Gamma distribution. (Hint: show the functions have the same form for some translation between $(\alpha, \lambda)$ and $(k, t)$ .)