Some background: A multilayer perceptron is the godfather of today's "AI". It roughly has the following structure as chained functions between vector space:
$$N(x)=f_n(...f_3(f_2(f_1(x))))=\hat{y}$$
where $x$ is typically a dataset and $\hat{y}$ a approximation to an relevant output. For example, data from a physics experiment $x$ mapped to a set of equations $\hat{y}$ that explains such data. This is done typically usually an algorithm called "back-propagation" which uses an "loss" function of $\hat{y}$ and some additional data $y$, these functions are usually distance functions in $\mathbb{R^m}$ which are differentiable and can be used with the chain rule to compute the derivative for the preceding $f_i$s. So here's my question:
Given a chain of functions $$f_i:\mathbb{R^u_i} \to \mathbb{R}^m_i$$
with composability: $$\mathbb{R}^u_{i+1}=\mathbb{R}^m_i$$ there should be a "space of differentiable functions" on $\mathbb{R}^u_1\to \mathbb{R}^m_{n}$. Is it possible to find a "basis" in the sense that their composition in this way "span" the entirety of this space?
Here's my thinking so far:
A derivative of a function $$f:\mathbb{R}^n\to\mathbb{R}^m$$ is a linear map $$d:\mathbb{R}^n\to\mathbb{R}^m$$ s.t. $$\lim_{h\in \mathbb{R}^n \to 0}\frac{|f(a)-f(a+h)-d(h)|}{|h|}=0$$ So $d$ and $f$ have basically the same "basis", so if you can find such a "basis" for $f$ you may be able to find it for $d$. My use of the word "basis" is based on my basic understanding of the Fourier transform is like giving a basis to the function space $\mathbb{R} \to \mathbb{R}$. Are there analogous constructions of the Fourier in higher dimensions?