I watched an MIT video about the Euler number. There they figure it out as follows:
The exponential function should be a function that per definition has the property, that it equals to its derivative.
So if $x=0$
$e^{x} = 1$, so the derivative 1 too. But then $e^{x}$ must be $1 + x$, but then the derivative too, but then $e^{x}$ must be $1 + x + \frac{1}{2}x^{2}$ and so on.
Why can we keep the sum of the previous results?
So why wouldn't be ok, if we would just say:
$e^{x}$ = 1, then the derivative is 1 too, but then $e^{x}$ must be !! x !! simply. So we wouldn't start the series.
I hope I am clear enough.
Here is the part of the video, it takes only 1 minute, so you can see better what I mean. At 7:11
At the start it says that it is assumed that at $x=0$ the value is $1$, so you can not take $x$ as it is not $1$ at $x=0$.