Let's define $\exp(z)$ in the following form:
$$\exp(z) = \sum \limits_{n=0}^{\infty} \frac{z^n}{n!}, z \in \mathbb{C} \tag{1}$$
We know that the definition above stands (I mean that the series converges).
We are to prove that $z\mapsto \exp(z)$ is a holomorphic function over $\mathbb{C}$.
Firstly I would like to prove that $\exp \in H(\mathbb{C})$.
I know that $f(z)=z$ is a holomorphic function (I've proven it using Cauchy-Riemann's equations).
It is obvious that $\forall_{n \in \mathbb{N}}$ $f_n(z) = \frac{z}{n}$ is holomorphic (is it?).
If the statement above stands it would be good to say that an infinite sum of holomorphic functions is holomorphic. I'm not sure it it's true however.
Secondly let's calculate $\frac{\mbox{d}}{\mbox{dz}}\exp(z)$.
Because we know that $z\mapsto \exp(z)$ is holomorphic (to my mind it basically means that I can calculate this derivative using methods from real analysis, am I correct?) and the series converges we can calculate it in the following form (using $(1)$):
$$\frac{\mbox{d}}{\mbox{dz}}\exp(z) =\frac{\mbox{d}}{\mbox{dz}} \sum \limits_{n=0}^{\infty} \frac{z^n}{n!} = \sum \limits_{n=0}^{\infty} \frac{\mbox{d}}{\mbox{dz}}\frac{z^n}{n!} = \sum \limits_{n=1}^{\infty} n \frac{z^{n-1}}{n!} = \sum \limits_{n=1}^{\infty} \frac{z^{n-1}}{(n-1)!} = \sum \limits_{n=0}^{\infty} \frac{z^{n}}{n!} = \exp(z)$$Is it calculated well?
You must know that every analytic function (i.e. which can be written as the sum of a power series, like $\exp$) is holomorphic.
Moreover, if you recall the theorem saying that it's possible to exchange $\displaystyle \sum_{n=0}^{+\infty}$ and $\dfrac{d}{dz}$ in the case of power series, then your computation is correct.
In any case, be careful to check first the radius of convergence of the power series. Here, using ratio test:
$$\dfrac{\dfrac{z^{n+1}}{(n+1)!}}{\dfrac{z^{n}}{n!}} = \dfrac{z}{n+1}$$
goes to $0$ when $n$ go to $+\infty$, thus it's convergent on $\Bbb C$.