Correct way to calculate numeric derivative in discrete time?

Question

Given a set of discrete measurements in time $x_t, t \in \{0,\Delta t, 2\Delta t,\ldots,T-\Delta t,T\}$, what is the correct way to compute the discrete derivative $\dot x_t$. Is it more correct to take the difference with the previous value: $$\dot x_t = \frac{x_t-x_{t-1}}{\Delta t}$$ or with the next value in time: $$\dot x_t = \frac{x_{t+1}-x_t}{\Delta t}$$

It may be that both are correct in different contexts, but I'm not sure which is appropriate when.

Likewise, how should the second derivative be computed: $$\ddot x_t = \frac{\dot x_{t+1}-\dot x_t}{\Delta t}$$ or $$\ddot x_t = \frac{\dot x_t-\dot x_{t-1}}{\Delta t}$$

For a fairly smooth discrete function and small $\Delta t$ both methods are roughly the same, but is one more correct than the other?

Answer 1

One way to go about computing a derivative that has second-order, rather than first-order, error is to use a centered difference:

$$\dot{x}_t = \frac{x_{t+1} - x_{t-1}}{2 \Delta t}$$

By 2nd-order error, I mean the error is $O(\Delta t)^2$ rather than $O(\Delta t)$.

At the boundaries, use a forward or backward difference as you provided.

The 2nd derivative is naturally centered:

$$\ddot{x}_t = \frac{x_{t+1} - 2 x_t + x_{t-1}}{(\Delta t)^2}$$

Answer 2

If you look for "finite-difference approximations" in any book on introductory numerical analysis. You will find that there are several so-called forward, backward and centered formulae for both first and second derivatives. The formulae you suggest for first derivatives are the backward and forward (respectively) approximations. They have order 1, which means that the difference between the actual value of the derivative and its approximations, $$ \left|\dot{x}_t - \frac{x_{t+1} - x_t}{\Delta t}\right| = O(\Delta t). $$ Here, I use $\dot{x}_t$ to denote the actual derivative of $x$ at $t$. This is all assuming that there is an underlying smooth function $x(\cdot)$ such that $x_t = x(t)$, i.e., your data is a sampling from a smooth process.

You can't really say that it is "more correct" to use the forward or backward finite difference; they both have order 1. This means that should you repeat your sampling with a $\Delta t$ that is half of what it currently is, the error above would be roughly halved as well.

As another poster mentioned, it is popular to use centered finite differences when possible because it is an approximation of order 2: $$ \left|\dot{x}_t - \frac{x_{t+1} - x_{t-1}}{2\Delta t} \right| = O(\Delta t^2). $$ This time, should your $\Delta t$ be halved, your error would be divided by (roughly) a factor of 4.

Choosing between approximation formulae is not always totally obvious. Clearly, the order of the error term is important but other factors could also come into consideration. For example, suppose your data is not generated beforehand but there is a cost associated to generating $x_{t+1}$ from the previously generated $x_s$, $(s \leq t)$. The backward scheme then esstentially comes for free while forward or centered schemes come at a cost.

The same considerations apply to approximations of second-order derivatives.

Those formulae are typically found by computing a simple function that interpolates your data (a polynomial or a spline, say), and then differentiating that simple function. The approximations you mention come from polynomial interpolation.