"Taylor Series" analog for functionals?

Question

For a function $f(x)$, it is possible to write it as a taylor series centered around a point $x=a$:

$$f(x)=\sum_{n=0}^{\infty}\frac{f^{(n)}(a){(x-a)}^{n}}{n!}=f(a)+f'(a)(x-a)+\frac{f''(a)(x-a)^{2}}{2}+...$$

(Of course, there's a lot more mathematical nuance to Taylor Series expansion, I just want to lay it out loosely here as a basis for my intuition.)

I'm wondering if there's anyway to apply this to functionals, that is, a functional $F[f(x)]$ that maps the function $f(x)$ to an output. Is there a way to "rewrite" a functional as a series such as this:

$$F[f(x)]=a_0+a_1(f(x)-\phi(x))+a_2(f(x)-\phi(x))^2+...$$

Where $\phi(x)$ is a function that acts analogously to the point $x=a$ in a Taylor expansion.

(Again, I'm using all of my terminology and notation pretty loosely here. I'm not going for robust mathematical rigoorousness; I just want to express my intuition behind this idea.)

Is this "Functional expanded as a series" idea a thing? What is it called? Does it have any applications?

Answer 1

Indeed there is. This is used in calculus of variation. Commonly up to and including order two. See https://en.wikipedia.org/wiki/Functional_derivative

Answer 2

It seems counter intuitive, but if you forget functionals (That is functions from functions to numbers) and just look at functions from functions to functions) you can build a taylor series like idea somewhat intuitively. I'm still working out the details of it, but i have been able to use it to show for example

$$ f(x+1) = f + f' + \frac{1}{2!}f''+\frac{1}{3!}f''' + ... $$

See here: How to build taylor series for infinite dimensional objects? for how to construct such series and for a list of problems that I encounter.

Somehow, most ideas in calculus generalize very cleanly in the context of "functions of functions" but fail to generalize EASILY (not saying they dont at all) in the context of functionals.

Answer 3

The expansion of a functional around a function is drastically more complicated than suggested in the question and in other answers here so far.

Wheras the Taylor series of $f(x)$ around point $x_0$ is written simply as:

\begin{alignat}{2} f(x) &= \sum_{n=0}^\infty c_n(x-x_0)^n &~~~,~~~c_n = \frac{f^{(n)}(x_0)}{n!}\tag{1}~, \end{alignat}

a Volterra series expansion of $f[x(r)]$ around the function $x_0(r)$ is significantly more complicated, as it requires integration over a dummy-variable version of $r$, an additional time for each successive term in the series):

\begin{align} f[x(r)] &= \sum_{n=0}^\infty \int \int \cdots \int c_n\left(\prod_{i=0}^n \left(x(r)-x_0(r^{(i)})\right) \right) \textrm{d}r^\prime \textrm{d}r^{\prime \prime} \cdots \textrm{d}r^{(n)}\tag{2}\\ &=\int \sum_{n=0}^\infty c_n \prod_{i=0}^n \left(x(r)-x_0(r^{(i)})\right) \textrm{d}r^{(i)}\tag{3}\\ c_n &=\frac{f^{(n)}[x_0(r)]}{n!}\tag{4}\\ &= \frac{1}{n!}\frac{\delta f[x_0(r)]}{\delta f[x(r^\prime)]\delta f[x(r^{\prime\prime})]\cdots \delta f[x(r^{(n)})]}. \tag{5}\\ \end{align}

Eq. 2 has been used in practical applications, for example in this 1994 paper in which $r$ represented a 3-dimensional vector so each integral in Eq. 2 was in fact a triple integral. Chapter 5 of this classical field theory book (PDF) by Nicholas Wheeler also has a lot of information and analogies about the above expansions.

I'll write the expansion out for you one more time in a way that might make it even more clear to you how this expansion compares with the Taylor expansion with which you are probably very familiar:

$$\tag{6}{\small f[x(r)] = f[x_0(r)] + \int \frac{f^{(1)}[x_0(r)]}{1!}\left(x(r)-x_0(r^\prime\right))\textrm{d}r^\prime + \int\!\!\!\int \frac{f^{(2)}[x_0(r)]}{2!}\left(x(r)-x_0(r^\prime\right))^{(2)}\textrm{d}r^\prime\textrm{d}r{^{\prime\prime}} + \cdots,} $$

in which:

$$ \left(x(r)-x_0(r^\prime\right))^{(2)}\equiv\left(x(r)-x_0(r^\prime\right))\left(x(r)-x_0(r^{\prime\prime}\right)). $$

This expansion has been the subject of my first MathOverflow question (earlier today!) which was about how to extend the Volterra series to functionals of more arbitrary complex-valued functions much like the Laurent series extends the concept of a Taylor series to more arbitrary functions of complex variables:

• “Taylor series” is to “Volterra series” as “Laurent series” is to _________?

I also answered today my first MathOverflow question, which was a 2.5 year old unanswered question about how to extend the Pade approximant in this way:

• “Taylor series” is to “Volterra series” as “Pade approximant” is to _________?

Some other Mathematics.SE questions which may interest you are:

• Functional Taylor series
• Taylor series with functions as parameters (as opposed to variables)
• Taylor expansion of functional