I know that the series is equal to
$$ \sum_{n=0}^{\infty}e^{-nx}= \frac{1}{1-e^{-x}}$$
However, if I expand each exponential term into a Taylor series I get
$$ \sum_{n=0}^{\infty}e^{-nx}= \sum_{n=0}^{\infty}\sum_{k=0}^{\infty}\frac{(-nx)^{k}}{k!} $$
Now if I use $ \sum_{n=0}^{\infty}n^{k}=\zeta (-k) $ I would get that
$$ \sum_{n=0}^{\infty}e^{-nx} = \sum_{k=0}^{\infty} \frac{(-1)^{k} \zeta (-k)}{k!}x^{k}$$
Is this last expression correct in the sense of Taylor series?
The derivation as it stands is not valid. Even though Bilou06 was probably speaking about convergent series, the relation $\sum_n\sum_m a_{n,m}=\sum_m\sum_n a_{n,m}$ can also fail for regularized divergent series.
In this case the conclusion is false - an error term is needed. We have two formulas at hand:
$$\zeta(-n)=-\frac{B_{n+1}}{n+1}$$
$$\frac{t}{e^t-1}=\sum_{k=0}^\infty B_k\frac{x^k}{k!}.$$
Therefore,
$$\begin{array}{ll} \sum_{k=0}^\infty\frac{\zeta(-k)}{k!}(-x)^k & =\zeta(0)+\sum_{k=1}^\infty-\frac{B_{k+1}}{(k+1)}\frac{(-x)^k}{k!} \\ & =-\frac{1}{2}+\frac{1}{x}\sum_{k=1}^\infty B_{k+1}\frac{(-x)^{k+1}}{(k+1)!} \\ & =-\frac{1}{2}+\frac{1}{x}\sum_{k=2}^\infty\frac{B_k}{k!}(-x)^k \\ & =-\frac{1}{2}+\frac{1}{x}\left[\frac{-x}{e^{-x}-1}-\frac{B_1}{1!}(-x)-\frac{B_0}{0!}(-x)^0\right] \\ & =\frac{1}{1-e^{-x}}-\frac{1}{x}-1.\end{array}$$