I have $n$ real values $x_1,x_2,\ldots,x_n$ and $n$ real values $y_1,y_2,\ldots,y_n$; then I have a function $f(x,\boldsymbol\theta)$ from $\mathbb{R}$ to $\mathbb{R}$ and depending on $m$ parameters $\boldsymbol\theta\in\mathbb{R}^m$.
The curve fitting problem is, given $x_1,x_2,\ldots,x_n$ and $y_1,y_2,\ldots,y_n$, find $\boldsymbol\theta$ such that:
$\underset{\boldsymbol\theta}{\operatorname{argmin}} \displaystyle\sum\limits_{i=1}^n (y_i-f(x_i,\boldsymbol\theta))^2$
I know that, for example, in the case
$f(x,\boldsymbol\theta)=\theta_0+\theta_1\,x$
the problem can be solved in closed form, i.e. $\boldsymbol\theta$ can be found by means of a set of formulae like the following (taken from Wikipedia page about linear regression):
$\overline{x} = \tfrac{1}{n} \sum_{i=1}^n x_i$
$\overline{y} = \tfrac{1}{n} \sum_{i=1}^n y_i$
$\theta_1 = \frac{ \sum_{i=1}^{n} (x_{i}-\bar{x})(y_{i}-\bar{y}) }{ \sum_{i=1}^{n} (x_{i}-\bar{x})^2 }$
$\theta_0 = \bar{y} - \theta_1\,\bar{x}$
What assumptions about $f$ do I need to make to get the existence of a closed form solution?
And I would like to ask the same above question for this problem too:
$\underset{\boldsymbol\theta}{\operatorname{argmin}} \displaystyle\sum\limits_{i=1}^n |y_i-f(x_i,\boldsymbol\theta)|$