There’s one point about the order of regularization that I can’t understand. Let’s suppose that we have an ill-posed problem $Af=g$, where $g$ is the observation (with noise), $A$ is a linear operator, $f$ is the input we need to find. Let’s assume that our solution should be smooth. Then we’re going to use the first order Tikhonov’s regularization which implies minimization of the following functional
$ ||Af-g||^2+\alpha ( ||f||^2+ ||\frac{df}{dx}||^2) $ (let’s consider 1D case)
So my question is about the units. Here we have a $\frac{df}{dx}$, the rate of change, while other members are of the “distance” units. It confuses me a little. I suggest that maybe the key idea of the methods is considering everything as a length, so $||\frac{df}{dx}||^2$ is the “length” of the derivative, but I’m not quite sure. Please help me to make this point clear. Thanks!
The theory of Tikhonov regularization doesn't really account for the dimensions/units of the quantities in the functional that is being minimized. In practice, if multiple regularization terms are being used, then weights are added to control the trade-off between the different terms. Thus you'd minimize
$\min \| Af-g \|^{2}+\alpha \| f \|^2 + \beta \left\| \frac{df}{dx} \right\|^2$.
It is possible to deal with inversion in a dimensionally consistent way by formulating the inverse problem in Bayesian terms. Here the regularization functional becomes a likelihood and everything is dimensionally consistent. However, in order to do this, you have to create a prior for the model parameters that takes into account their dimensions.