I have partially understood why. The two major reasons stated are:
Time elapsed cannot be negative
The cumulative distribution $P(X \leq x)$ increases monotonically and asymptotically approaches 1.
Point 1 implies that the probability density function $p(x)$ should be 0 for all $x < 0$. Point 2 makes sense that as time increases, it is more likely the event will come to an end. But are there no other distributions that satisfy these criteria?
Is there a way to check the validity of this assumption in some real cases. Say, I have some duration data of a process, how can I check if it is drawn from an exponential distribution?
I hope I am not asking two different questions or something.
The exponential distribution satisfies the memoryless property, which is sometimes a reasonable modeling assumption. Also, often a situation where rare events are happening at random can be modeled as a Poisson process, and you can show that the waiting times between events in a Poisson process are exponentially distributed.