Can we approximate delayed-differential equations with higher-order-ordinary-differential-equations?

Question

I noticed that the most simple numerical approximation of a higher order-differential equation has the same form as the numerical approximation of a delayed first-order differential equation. This leads me to the following hypothesis:

Hypothesis: Delayed first-order differential equations can be approximated by higher-order ordinary differential equations.

I wanted to investigate to what extent this is true so I came up with a method to find approximations of delayed differential equations. However, my initial tests seem to indicate that this approach doesn't work. I hope someone can explain why, and if there is a better, similar approach.

Here is my approach:

Take the simple delayed first-order differential equation for arbitrary function $f$: $$\dot x(t+1)=f(x(t))$$ By taking $\Delta x =1$, this is numerically approximated by the difference equation: $$x_{t+2}-x_{t+1}=f(x_t)$$ adding to both sides $x_t-x_{t+1}$, gives: $$x_{t+2}-2x_{t+1}+x_t=f(x_t) - (x_{t+1}-x_t)$$ Which, if we again take $\Delta x =1$, is the numerical approximation of $$\ddot x (t)=f(x(t))-\dot x(t)$$

Hence we might approximate a simple first-order delayed equation by a higher order non-delayed equation: $$\dot x(t+1)=f(x(t)), \quad \text {is approximated by}\quad \ddot x(t)+\dot x(t)=f(x(t)) $$ By the same approach we could show that $$\dot x(t+2)=f(x(t)), \quad \text {is approximated by}\quad \dddot x(t)+2\ddot x(t) +\dot x(t)=f(x(t))$$ $$\dot x(t+3)=f(x(t)), \quad \text {is approximated by}\quad \ddddot x(t)+3\dddot x(t) -3\ddot x(t)+\dot x(t)=f(x(t))$$ And so forth...

However, I've been using $\Delta x =1$ here. If we take $\Delta x=1/n$, and gradually increment $n$ upwards, a similar pattern to the one for larger delays occurs: $$\dot x(t+1)=f(x(t))$$ is approximated for $n=2$ by: $$2\cdot (x(t+1+\frac {1}{2})-x(t+1))=f(x(t))$$ using the same approach as above, but $\Delta x = \frac{1}{2}$ instead of $\Delta x = 1$, and a lot of tedious algebra, one can show that this is equivalent to the approximation of
$$2^{-2}\dddot x(t)+2^{-1}\cdot 2 \ddot x(t)+2^{-0}\dot x(t)$$

Hence we might approach a closer and closer approximation of $\dot x(t+1)=f(x(t))$, by taking increasingly larger $n$, and smaller $\Delta x=\frac{1}{n}$, as follows:

using $n=2$ (resulting in $\Delta x=\frac {1}{2} $): $$\dot x(t+1)=f(x(t)), \quad \text {is approximated by}\quad n^{-n}\cdot \dddot x(t)+n^{-n+1}\cdot 2 \ddot x(t)+n^{-n+2} \cdot \dot x(t)$$ and using $n=3$: $$\dot x(t+1)=f(x(t)), \quad \text {is approximated by}\quad $$ $$n^{-n}\cdot \ddddot x(t) +n^{-n+1}\cdot 3\dddot x(t)+n^{-n+2} \cdot 3\ddot x(t)+ n^{-n+3}\cdot \dot x(t)$$

And so forth...

The form of this approximation is the same as for $\Delta x =1$, but with higher delays, except for the coefficients of $n$.

The most important point I noticed about this approximation is that as we increase $n$, increasingly higher order derivatives are added, but the coefficients of those derivatives seem to decrease hyperexponentially (i.e. $n^{-n}$). This made me hope that perhaps the approximations would converge quickly to the delayed equation as $\Delta x$ decreases.

However, I did some tests on the delayed equation $\dot x(t)=x(t-1)$, and $\dot x(t) = x^2(t-1)$, where my approach fails miserably. increasing $n$ actually deteriorates the approximation.

On the one hand (in hindsight) this makes sense to me, since increasing the order of the derivative should increase the rate of growth of $x$ for large $t$, but it still bugs me that it doesn't work, despite the fact that the difference equations for the two are so similar.

So my question is: Why doesn't my approach work, given that the difference-equation approximations of delayed-differential-equations, and higher-order-differential equations have the same form? and more importantly, is there a different but similar approach to approximate delayed-differential-equations?

Answer 1

If looking for a solution for $\dot x(t+1)=f(x(t))$ on $[0,\infty]$, take any continuous "seed" function $x:[-1,0]\to\Bbb R$ with $x(0)=x_0$ and define the solution as $$ x(t)=x_0+\int_0^t f(x(s-1))ds $$ The values on $[0,1]$ are well-defined using the values on $[-1,0]$, the values on $[1,2]$ by the values on $[0,1]$ and consequently $[-1,1]$ etc. By construction this function is continuous and by the integration it is differentiable on $(0,\infty)$.

As you can see, you get lots of solutions that satisfy this delay differential equation. Before starting to approximate, you will have to define which of them exactly you want to approximate.

Answer 2

A delay can be written as a differential $e^Dx(t)=x(t+1)$. So it might make sense to approximate $$\dot{x}(t+1)=f(x(t))$$ by cutting off the following series at some $n$, $$e^DDx(t)=f(x(t))$$ $$(\cdots+\frac{D^n}{(n-1)!}+\cdots+D^2+D)x(t)=f(x(t))$$ This ties in with Lutz Lehmann's answer in that an infinite number of initial data is required to specify the solution.

For example, approximate $\dot{x}(t+1)=f(x)$ by $(\frac{D^5}{4!}+\cdots+D^2+D)x(t)=f\circ x(t)$.