Numerical compuation of functional derivative

Question

I was wondering if it is possible to numerically compute $$\frac{\delta L}{\delta f}$$ where $L$ is a functional of the function $f$ known only in a discrete number of point by numerical computation of $L(f)$. Looking online I found few resources among which, this answer on SE. Following the ideas in that answer I tried a numerical test.

1. I have a discrete function $f_i$ and functional $L(f_i)$ known in every point $i$ of my mesh.
2. So I brutally apply first order finite difference to compute $$\left.\frac{\delta L}{\delta f}\right|_i=\frac{L(f_{i+1})-L(f_{i})}{f_{i+1}-f_{i}}$$

For simple test case ($f=\cos(x),\log(x),\exp(x)$ and $F=\cos(f),\sin(f),\log(f)$), the algorithm seems to work fine. Now the question, is this only a coincidence or this kind of computation for the functional derivative is consistent for every case ?

Answer 1

If $L$ depends only on $f$ at the point, i.e., $L(f)=\int_I F(f(x))\,\mathrm{d}x$ then its functional derivative is $F'(f(x))$ so your method works. For more complicated $L$ your method breaks down, e.g., $L(f)=\int_{-1}^1 f(x)f(-x)\,\mathrm{d}x$ or $L(f)=\int_I f'^2$.

Answer 2

You forget that functional depends on the function and not the coordinate. In other words, you are supposed to vary the function itself, not its argument.

For example, in physics we have action which depends on the trajectory between the two points. You can have an infinite number of different curves connecting the points, and to find the variation, we consider two paths which are very close to each other (according to some definition) and see how the functional changes between them. Again, we are not varying the arguments of the functions, we are varying the functions themselves.

Thus, for a general functional, I doubt there's a universal method to numerically approximate functional derivatives.

However, we can do it for certain classes of functionals. The first class is represented in your answer in examples you listed. It's just a function of $f$, like $f$ is a function of $x$. (Or actually, I'm not sure if those examples are even functionals, they are rather composite functions).

---

Another important class of functionals (where the physics action also belongs):

This functional is a number, for example, the length of a curve between $a$ and $b$:

$$L(f)=\int_a^b \sqrt{1+(f'(x))^2}dx$$

Here $f(x)$ can be any function for which the integral exists, and the number $L(f)$ depends on it in a certain way.

How do we find the derivative here?

In the case of functionals in the form:

$$L(f)=\int_a^b G(x,f,f')dx$$

We have, the Euler-Lagrange equation, which states:

$$\frac{\delta L}{\delta f}=\frac{\partial G}{\partial f}-\frac{d}{dx} \frac{\partial G}{\partial f'}$$

This one is ready for numerical implementation, if the explicit form of the three variable function $G$ is known. We can use finite differences here.

(Though, considering we have two partial derivatives and one total derivative, setting it up will be very complicated in the general case).