Given an operator $O(f(z)): \left(\mathbb{C} \rightarrow \mathbb{C}\right) \rightarrow \left(\mathbb{C} \rightarrow \mathbb{C}\right) $
We can define its functional derivative up to a perturbation $\lambda$ as
$$ \delta_\lambda O = \lim_{\epsilon \rightarrow 0} \frac{O(f+\epsilon \lambda) - O(f)}{\epsilon} $$
This can be (with a slight abuse of notation expressed as)
$$ \delta_\lambda O = \frac{\partial O}{\partial f} \lambda + \frac{\partial O}{\partial f'}\lambda ' + ... = \sum_{n=0}^{\infty} \frac{\partial O}{\partial f^{(n)}}\lambda^{(n)} $$
We will adjust this notation to the leibniz style (for reasons that will be obvious in the next next step):
$$ \delta_\lambda O = \frac{\partial O}{\partial f} \lambda + \frac{\partial O}{\partial \frac{df}{dz}}\lambda ' + ... = \sum_{n=0}^{\infty} \frac{\partial O}{\partial \frac{d^nf}{dz^n}} \frac{d^n \lambda }{dz^n} $$
You then get some nice identities (I call them the taylor extractants):
$$ \frac{\partial O}{\partial f} = \delta_{1}O \\ \frac{\partial O}{\partial f'} = \delta_xO-x\delta_1 O \\ \frac{\partial O}{\partial f''} = \delta_{\frac{1}{2}x^2}O-x\delta_{x}O+\frac{1}{2}x^2\delta_1O$$
etc...
Now it would be nice to assume that all such operators $O$ can be written as an infinite sum of terms of the form
$$a(z) f^{n_1}\left(\frac{df}{dz}\right)^{n_2}\left(\frac{d^2f}{dz^2}\right)^{n_2}... = a(z) \prod_{k=0}^{\infty}\left[ \left( \frac{d^k f}{dz^k} \right)^{n_k}\right]$$
Much like how all holomorphic functions can be written as an infinite sum of terms of the form
$$ a_n z^n$$
We then get a definition of an "infinite dimensional taylor series" as a series of the form
$$O(f) = \sum_{g \in \left( \mathbb{N}_0 \rightarrow \mathbb{N}_0 \right)} \left[ \mathfrak{a}(g) \prod_{k=0}^{\infty}\left[ \left( \frac{d^k f}{dz^k}\right)^{g(k)} \right] \right]$$
Where $\mathfrak{a}: (\mathbb{N}_0 \rightarrow \mathbb{N}_0) \rightarrow (\mathbb{C} \rightarrow \mathbb{C})$ assigns complex function coefficients to our terms, which are products of the derivatives of different orders of $f$. This is a direct generalization of the usual expression of a taylor series as
$$ f(z) = \sum_{k \in \mathbb{N}_0} a_k z^k $$
With this set up, I was hoping then to try to decompose the operator $O(f) = f(f)$ (ex: it sends $x^2$ to $x^4$ and $e^z$ to $e^{e^z}$) into this taylor series framework. But I unfortunately ran into some problems. It seems that besides the expressions $\frac{\partial^n O}{ \partial f^n}$ all other "functional-partial-derivatives" including the mixed derivative terms equal $0$. Which means that my infinite dimensional taylor series for $f(f)$ CANNOT be made to converge except on a single function. I.E. it offers no utility in extrapolating the value of $f(f)$ to new functions.
Now this isn't super shocking to me, because sometimes taylor series have radius of convergence zero, see for example $e^{-\frac{1}{x^2}}$. So I'm guessing that $f(f)$ just happens to have $0$-radius of convergence infinite dimensional space but i'm not ENTIRELY sure if that might mean I forgot to look at it from the correct angle (much like how $e^{-\frac{1}{x^2}}$ has a useful laurent series, could it be that f(f) has a useful infinite-dimensional-laurent series/some other generalization thereof?)
Has anyone else worked with/seen similar infinite-dimensional taylor series, and am I potentially missing important terms here?
Visualization tool:
Visualizing the infinite dimensional sum is not easy, as you are attempting to visualize a commutative free group with countably infinitely many generators. One approach for "seeing" the picture is via the partition sum representation
$$ O = c_0(z) + a_0(z)f \begin{matrix} a_1(z)f' \\ + \\ a_2(z)f^2 \end{matrix} \begin{matrix} a_3(z) f'' \\ + \\ a_4(z) ff' \\ + \\ a_5(z)f^3 \end{matrix} + \begin{matrix} a_6(z) f''' \\ + \\ a_7(z) f''f \\ + \\ a_8(z)f'f^2 \\ + \\ a_9(z)(f')^2 \\ + \\ a_{10}(z)f^4 \end{matrix} + ... $$
Where the height of each column is equal to the number of partitions (in the combinatorial/number theoretic sense) of the index of the column.
Update:
It turns out that I made a mistake. Seeing the ramifications now. FWIW I was able to successfully derive $f(x+1) = f + f' + \frac{1}{2}f'' + \frac{1}{6}f''' + ... $ using this framework
So far:
Unforunately still stuck
Observe that for the case of $O(f) = f(f)$ we have
$$ \delta_\lambda O = \lim_{\epsilon \rightarrow 0} \frac{f(f+\epsilon \lambda)+\epsilon\lambda(f+\epsilon \lambda)-f(f)}{\epsilon} = f'(f)\lambda+\lambda(f)$$
$$ \frac{\partial O}{\partial f} = \delta_1 O = f'(f)+1$$
$$ \frac{\partial O}{\partial f'} = \delta_x O - x\delta_1O = xf'(f)+f - x(f'(f)+1) = f-x$$
$$ \frac{\partial O}{\partial f''} = \delta_{\frac{1}{2}x^2} O - x\delta_xO + \frac{1}{2}x^2\delta_1O= \frac{1}{2}x^2f'(f)+\frac{1}{2}f^2- x^2 f(f)-xf+\frac{1}{2}x^2f'(f)+\frac{1}{2} = \frac{1}{2}(f-x)^2$$
Generally then we can extract the rest of the terms this way so we find that
$$ f(f) =_{expected} (f'(f)+1)f + (f-x)f'(x) + \frac{1}{2}(f-x)^2f''(x) + ...$$
But in reality we all know
$$ f(f) = f(x) + f'(x)(f-x)+ \frac{1}{2}f''(x)(f-x)^2 + ... $$
So something here is quite off.