Linear regression is very well known and widely used.
What it means is basically if we choose a set of functions to do linear regression with, (let's call them $\{\phi_1,\phi_2, \cdots, \phi_N \}$):
$$f(x) = \sum_{k=1}^N c_k\phi_k(x)$$
What the objective is to choose the $c_k$ to make this sum of $\phi_k$s fit some function $f(x)$ as well as possible for a set of $x$. In linear regression this as well as possible is the 2-norm: $${\bf c_o}=\min_{\bf c}\|{\bf f-\Phi c }\|_2^2$$
Where we have stuffed
- the $f(x)$ values in the column vector $\bf f$,
- the $\phi_k(x)$ values as column vectors into $\bf \Phi$ and
- the $c_k$ are stuffed in a column vector $\bf c$
This can be solved by linear algebra and linear least squares.
Now to a more complicated problem, we want to fit $\phi\{p\}(x)$ where $p$ is a parameter argument. For example $$\phi\{{\bf p}\}(x) = \exp(p_0 + p_1x+p_2x^2)$$
Now many of us probably know how to turn this one into a linear problem (how?). But how would we build a computational machinery to do turn it into a linear problem in general for a completely generic function which is non-linear in parameter argument $\bf p$ ( and in $x$ )?
You could apply nonlinear regression by using the least squares approximation. But you will not be able to linearize the expression for any given function.
In your given example one could linearize the problem by noting that $\phi = \exp(p_0x^0+p_1x^1+p_2x^2) \implies \ln{\phi}=p_0x^0+p_1x^1+p_2x^2$. Hence, if you transform your dependent variables to be $y=\ln\phi$ and your independent variables as $1=x^0=x_0$, $x=x_1$, $x^2=x_2$ and so forth then you have transformed the problem into a multiple linear regression which can be solved analytically for the least squares approximation by using the Moore–Penrose pseudoinverse.
In some cases with $\phi(x)=f(x;\mathbf{p})$ in which $\mathbf{p}$ is a vector of parameters, you could try a taylor expansion to the $k^{\text{th}}$ of $\phi(x)$ such that you have
$$\phi(x)\approx f(x;\mathbf{p})\bigg|_{x=x_{0}}+\dfrac{df(x;\mathbf{p})}{dx}\bigg|_{x=x_{0}}(x-x_0)+\cdots+\dfrac{d^kf(x;\mathbf{p})}{dx^k}\bigg|_{x=x_{0}}(x-x_0)^k.$$
Using the transformation $x_k=(x-x_0)^k$ will turn your problem into a multiple linear regression. You will need to choose an evaluation point $x_0$ as well as the order of the Taylor approximation. For data analysis, one should not use very high order polynomials as you might run into overfitting issues by doing so.
You could also try different series expansions e.g. Fourier series for periodic signals or Padé Series for functions with singularities.