Least Squares two different forms for the residual

46 Views Asked by At

Least squares residuals are given in the basic form:

$$r_i = y_i - f(x_i, \theta)$$ where $y_i$ is the observation and $f(x_i,\theta)$ is the value predicted by the model. I came across another form as follows

$$ r_i = y_{t_i} - y_{t_{i-1}} - f(x_{t_{i-1}}, \theta)\Delta_{t_{i-1}} $$ where $t_k$ is the time of observation, $y_k$ is the observation and $f$ is as before, and $\Delta_{t_{i-1}}$ is the difference in times $t_i$ and $t_{i-1}$.

I feel quite daft at the moment for not immediately seeing why theres a reason to use this second form. However, thought perhaps someone here could enlighten me, perhaps with an intuitive or rigorous explanation.

1

There are 1 best solutions below

1
On BEST ANSWER

I think it needs to be clarified in the question what the model is before this can be answered with certainty. Based on your comment about the observations arising from a discretization, I think the residual in your answer is based on an argument that is similar to the following:

Consider the initial value problem $$ \frac{dy}{dt} = f(y,t), \qquad t_0 \le t \le T, \qquad y(t_0)=y_0. $$

If you wanted to evaluate this on the grid

$$ t_0 < t_1 < t_2 < \dots < t_n < T $$ with $t_n = t_0 + nh$ where $h$ is the step-size. For a single step, Euler's method gives

$$ y_{i+1} = y_i + h f(t_i, y_i). $$

where $h=t_{i+1}-t_i$. So if you were to predict $y_{i+1}$ from knowledge of $y_i$ and $f$, then your residual would just be the difference in these two quantities.