Overdetermined system of quadratic equations

621 Views Asked by At

In second order polynomial least squares one has a set of observations $(x_i,y_i), 1 \leq i \leq m $, and, using a second order polynomial, one has to find the coefficients $ a_0, a_1, a_2 $, since the residual is $ R = \sum_{i=1}^m [ y_i - (a_0 + a_1 x_i + a_2 x_i^2 ) ] ^2 $, and the solution is $\mathbf{a}=(X^TX)^{-1}X^T \mathbf{y}$, where $X$ is a Vandermonde matrix.

I am facing a slightly different problem. I obtain a set of observations $(a_{0i},a_{1i},a_{2i},y_{i})$, and I am trying to minimize the residual $ R = \sum_{i=1}^m [ y_i - (a_{0i} + a_{1i} x + a_{2i} x^2 ) ] ^2 $, and obtain a solution for $x$. It's a kind of an overdetermined system of quadratic equations, if I am not wrong. I also noticed that the system can be written in the form of $ \mathbf{y} = A \mathbf{x} $, where $ \mathbf{x} = [1 \quad x \quad x^2]^T$, i.e. a Vandermonde matrix of order 3, having only 1 row (hence, it's a Vandermonde - looking vector, if I can use this term).

My question is: Is there a way to obtain a straightforward solution to this problem (for instance, in a matrix form, like in linear least squares), without using any recursive algorithms?

1

There are 1 best solutions below

1
On

$R$ is a polynomial of degree 4 in $x$. The coefficient of $x^4$ is $\sum_{i=1}^ma_{2i}^2>0$. It has one or two local minima, one of which is the global minimum. To find them, you solve the third degree equation $$ \frac{dR}{dx}=-2\sum_{i=1}^m \bigl( y_i - (a_{0i} + a_{1i} x + a_{2i} x^2 )\bigr)(a_{1i}+2\,a_{2i\,}x). $$ There are formulas for the solution of third degree equations, but it might be simpler to use a numerical method.