Intuitive explanation of $y' = y \implies y = Ce^x$

Question

I understand why $f : \mathbb{R} \to \mathbb{R}$ with $f'(x) = f(x)$ and $f(0) = 1$ must be $f (x) = e^x$, but I don't really feel it is super intuitive. Intuitively, why would you expect such a function to satisfy $$f(a)f(b) = f(a+b)$$ or have exponential growth?

To build intuition, I tried the discrete case first, i.e.,

$$\frac{f(x+h) - f(x)}{h} = f(x) \implies f(x+h) = f(x)(h+1)$$

so

$$f(y) = f(0) (1+h)^{\frac{y}{h}} = f(0) c_h^{y}$$

where $c_h = (1+h)^{\frac{1}{h}}$ and saw what happens when $h \to 0$. However, I'm interested in a more intuitive explanation, if there's one. Bonus if it also explains $y' = P'y \Rightarrow y = Ce^P$ in a nice intuitive way.

Answer 1

Depending on how familiar you are with Calculus, you would find that the most important intuitive marker for this property $f'(x)=f(x)$ or $\frac{dy}{dx} \implies y=e^x$ is that the derivative of the exponential function is the exponential itself. Provided that you are familiar with the derivation (such as it is), the following might prove helpful-

You know that the slope of the function $f(x)$ that you are dealing with equals it's value at the point.

Think, what does this imply ?

Case - 1

The slope if 0 at a point where the function takes value 0. This is the straight line $y=0$.

Case - 2

The function takes a positive value at some point. Then it must have a positive slope at that point, implying that it's value increases there. So, it's value keeps increasing and so does its slope- the rate of increase in its value.

What does that mean? That the value of the function keeps increasing at an ever increasing rate. Intuitively, this suggests exponential growth.

Case-3

Finally, consider the case where the function takes a negative value somewhere. Now, it is negatively sloped, and it's value decreases at an ever increasing (or should I call that decreasing?) rate.

Thus, there are $3$ cases- exponential increase and decrease and the singular solution on the x-axis. All these can be recovered explicitly by solving the original differential equation are varying bthe values of the constants involved (or equivalently, changing the initial conditions of the IVP).

Of course there are also the cases of exponential decay (they don't arise in this case, but can if the constant $k$ in $f'(x)=k\cdot f(x)$ is negative and so, thinking about these might prove helpful in gaining understanding) but they are intuitively similar, now the value decreases and so does its rate of decrease (or otherwise, just the above argument along the negative x direction).

In the end however, you simply need to get familiar with this - spend time solving problems and thinking about it. Solving the problem from first principles (using limits and Riemann sums) may help, but not much.

Answer 2

Let the solution be some function such that $$y(0)=y_0,y(1)=y_1,\\y'(0)=z_0,y'(1)=z_1.$$

Without loss of generality, we take $y_0=1,y_1=a$. By the ODE, we have $z_0=1$, $z_1=a$, as the slopes are proportional to the ordinates.

Now if we move to the interval $[1,2]$, we will ensure continuity of the function and of its derivative if we just multiply by $a$, i.e.

$$y(1)=a=y'(1),\\y(2)=a^2=y'(2).$$

More generally,

$$y(n)=y'(n)=a^n$$ is an exponential.

Answer 3

The answer to such a question depends crucially on what you are trying to relate to what. There are many "definitions" of the function $e^x$ and one can try to relate them to each other in intuitive way. Here is a (partial?) list:

1) The function $e^x$ is the unique solution of the ODE $f'(x)=f(x)$ with initial value $f(0)=1$. The high brow rephrasing is that $e^x$ is the eigenfunction of the operator of differentiation with eigenvalue 1, normalized (i.e. rescaled) so that $f(0)=1$.

2) The function $e^x$ is the differentiable function solving $f(a+b)=f(a)f(b)$ with $f'(0)=1$.

3) The function $e^x$ is given by convergent power series $\sum_{i=0}^{\infty} \frac{x^n}{n!}$.

4) The function $e^x$ is the inverse function of the function $\ln x$, which in turn is the function of positive $x$ whose derivative is $\frac{1}{x}$ and such that $\ln 1=0$.

5) The function $e^x$ is equal to $\lim_{n\to \infty} (1+\frac{x}{n})^n$.

Now one can try to relate the various definitions in various ways.

I think what you are asking is the intuitive reason why 1 implies 2. I think the finite difference approach you tried is somewhat helpful: if you replace the ODE with the finite difference version $f(n+1)-f(n)=f(n)$ (and still $f(0)=1$) you immediately see $f(n+1)=2f(n)$, $f(n)=2^n$. You can then try $f(n+1/2)-f(n)=\frac{1}{2}f(n)$, so $f(n+1)=(1+\frac{1}{2})^2 f(n)$. This is getting closer to what you did. This is the "yearly interest of 100%" model of investment, versus "50 percent interest every half year". The ODE version is a "continuous interest compounding" (limit of $100/n$ percent interest $n$ times per year). The relation between the two is the relation between approximately solving the ODE by Euler method and actually solving the ODE (see below).

I think the definition 1 is fundamental, with the rest secondary (but of course also very important). It is unfortunate that to make it into official definition would require establishing the theorem on existence and uniqueness of solutions to ODEs, which sometimes uses exponential function, so one has to be a bit careful. In any case, one could argue that conceptually it is the main definition.

In any case, now we show that all other definitions imply the first:

2 implies 1: To start, note that $f'(0)=1$ means that $f$ is not constantly zero function. Now plug in $a=0$ into $f(a+b)=f(a)f(b)$ to get $f(0)=1$.

For the main part, differentiate $f(a+b)=f(a)f(b)$ in $a$. Get $f'(b+a)=f'(a)f(b)$. Now plug in $a=0$, get $f'(b)=f(b)$. QED.

3 implies 1: We plug in $x=0$ to get $f(0)=1$, and we differentiate term by term to get $f'(x)=f(x)$.

4 implies 1: By the theorem on derivative of inverse function, if $f(x)$ is the inverse of $\ln x$, then $\ln' x=\frac{1}{x}$ means $f(x)'=1/\ln'(f(x))=\frac{1}{1/f(x)}=f(x)$; also $\ln 1=0$ means $f(0)=1$.

5 implies 1: The expression $(1+\frac{x}{n})^n$ is nothing else but the approximate value at $t=x$ of solution to $f'(x)=f(x)$ with $f(0)=1$ given by applying Euler method for $n$ steps with each step of size $\frac{x}{n}$. Indeed, we claim that after $k$ steps in the Euler method we have as current value $y_k=(1+\frac{x}{n})^k$. This is clearly true for $k=0$, and inductively, starting from $y_k=(1+\frac{x}{n})^k$ since $f'=f$ we are instructed to move at rate $v=(1+\frac{x}{n})^k$ for time $\Delta t=\frac{x}{n}$, so we end up at $y_{k+1}=y_k+ (\Delta t) v=(1+\frac{x}{n})^k+\frac{x}{n} (1+\frac{x}{n})^k=(1+\frac{x}{n})^{k+1}$. Thus, if we believe that Euler method converges to true solution as step size goes to zero then we conclude that indeed 5 implies 1.

This implies (via uniqueness of ODE solutions) that if a function satisfying any of 2-5 exists is is unique; existence is clear for 4 after one observes that $\ln'(x)=\frac{1}{x}>0$ implies that $\ln x$ is monotone increasing; it follows for 3 from establishing that convergence radius of the series is infinity; for 5, it follows from our proof (again, if we believe Euler approximation to converge); and it can be established for 2 for example by taking the function in 3 and verifying that if satisfies 2 by plugging in and using binomial expansion.

Answer 4

Obviously, $f' = f$ implies $f$ is $C^\infty$ and using almost any form of the remainder, the Taylor series $$\sum_{n=0}^\infty\frac{x^n}{n!}$$ will converge to the function. The property $f(x)f(y) = f(x + y)$ can be deduced from the power series. See Proof of homomorphism property of the exponential function for formal power series.

Answer 5

Let's prove that

Theorem: If $f:\mathbb{R} \to\mathbb {R} $ is a function such that $f'(x) =f(x) \, \forall x\in\mathbb {R} $ and $f(0)=1$ then $$f(a+b) =f(a) f(b) \, \, \forall a, b\in \mathbb {R} $$

First we prove that $f$ never vanishes. This is done by considering the function $g$ defined via $g(x) =f(x) f(-x) $. We have $$g'(x) =f'(x) f(-x) - f(x) f'(-x) =0$$ as $f'=f$. Therefore $g$ is constant and $$f(x) f(-x) =g(x) =g(0)=f(0)f(0)=1$$ for all real $x$. Thus $f$ never vanishes.

Next let $a\in\mathbb{R} $ and consider the function $F$ defined by $$F(x) =\frac{f(x+a)} {f(x)} $$ Clearly $$F'(x) =\frac{f(x) f'(x+a) - f'(x) f(x+a)} {f^2(x)}=0$$ as $f'=f$. Therefore $F$ is constant and $F(x) =F(0)=f(a)$ and therefore $$f(x+a) =f(x) f(a) $$ Replacing $x$ by $b$ we get the desired functional equation.

You should observe that both the differential equation $f'=f$ and initial condition $f(0)=1$ are necessary to prove the functional equation. Just having the differential equation $f'=f$ does not guarantee that $f(x) f(y) =f(x+y) $.

You may also have a look at this solution of $f'=f$ which does not assume anything about exponential function.

The theorem mentioned in this answer also holds when $\mathbb{R} $ is replaced by $\mathbb{C} $ but the proof requires the use of Taylor series (a link in the comments to question by user21820 deals with this).

Answer 6

All 1st order linear ODEs have the property $$ f (a) f (b) = f (a+b) $$ Some call it the semigroup property$^\color{red}{\star}$ of the state transition matrix:

---

---

Answering your questions:

• You have the ODE $\dot y = y$, which is 1st order and linear. Hence, your $1 \times 1$ state transition matrix has the semigroup property. Arguably, this is more interesting in higher dimensions.

• Exponential behavior (whether growth or decay) can be defined by the law $\dot x = \alpha x$, i.e., the rate of change at a certain time is directly proportional to the magnitude at that same time.

---

$\color{red}{\star}$ Panos Antsaklis, Anthony Michel, Linear Systems, Springer, 2006.

Answer 7

A couple of years later, here is an attempt to give a more direct treatment to the question actually asked: "why should we expect the function satisfying $f'=f$ and $f(0)=1$ to satisfy $f(a)f(b)=f(a+b)$ or have exponential growth?"

Of course, exponential growth happens for any solution of any $f'=Kf$ with $K>0$. Why?

Here is a basic intuition.

Suppose we start with some amount of stuff $f(0)$, and suppose that after some amount of time - say $T$ days - of evolving according to $f'=Kf$ the amount has doubled: $f(T)=2f(0)$. We can now mentally split the new pile of stuff into two piles of size $f(0)$. We observe that evolving the size of the original "combined" pile by $f'=Kf$ is equivalent to evolving each of the two sub pile sizes $f_1$ and $f_2$ by their own rules $f_1'=Kf_1$ and $f_2'=Kf_2$ -- after all, each of these ODEs simply says that the rate of growth is proportional to the amount already present. It's as if each "unit" of stuff is producing some fixed extra amount at every instance -- whether we view all units as part of the big pile of size $f$ or as belonging to one of the sub piles of sizes $f_1$ and $f_2$ is immaterial. (Formally, this is expressed by saying that the equation is "linear" -- if $f_1$ and $f_2$ are solutions, then so is $f=f_1+f_2$.)

But now each sub pile is starting from the same size $f(0)$ as the original pile was at the start time $t=0$. Now, since the evolution law is the same (the ODE is time-invariant aka autonomous), each sub pile will from then on grow in the same way as the original pile from $t=0$, doubling in size after time $T$. So the whole pile again doubles in size after another $T$ days. Then, after another $T$ days, it doubles again, etc.

So the solution of $f'=Kf$ doubles at regular intervals. This is exponential growth.

Now, some refinements:

1. Suppose $T$ is not the doubling time, but is instead, say, the time to reach $3/2$ of the original size. Let's then mentally splitt the original pile into two half-piles. Now, after time $T$ merge the results and resplit into 3 half-piles. After $T$ more days each half-pile grows by the factor of $3/2$ again. So we see that after each new period of $T$ days the amount of stuff will again get multiplied by $3/2$. By similar reasoning, for any positive rational $r=p/q$ the time it takes to get the amount of stuff to multiply by $r$ is always the same.

2. Of course, it is possible that the amount in the pile never doubles at all. We know that this is what in fact would happen if $K<0$. But if $K>0$ (and $f(0)>0$), then $f'(0)=Kf(0)>0$ so the amount will grow initially -- and then by 1) it will keep growing.

3. A slight refinement/modification/reformulation of this "splitting piles into sub piles and evolving each separately" argument says that for any time interval $T$ the ratio of the pile size after to the pile size before is the same. In formulas, we have $f(t_1+T)/f(t_1)=f(t_2+T)/f(t_2)$ for any $t_1$ and $t_2$. Taking $t_1=a, T=b, t_2=0$ we have $f(a+b)/f(a)=f(b)/f(0)$ aka $f(a+b)f(0)=f(a)f(b)$. If $f(0)=1$ this is the equality $f(a+b)=f(a)f(b)$ we were after.