Calculating coefficients for least squares [cross-posted from CrossValidated]

Question

In this blog, this author says to calculate the coefficients for the equation $$ Flat(x, y) = A + Bx + Cy + Dx^2 + Ey^2 + Fxy $$ using least squares.

I found this PDF that shows how to do the calculation for $$ z = Ax + By + C $$ using a matrix solution to linear equations as follows: $$ \left[ \matrix{ \sum_{i=1}^m x^2_i & \sum_{i=1}^m x_iy_i & \sum_{i=1}^m x_i \cr \sum_{i=1}^m x_iy_i & \sum_{i=1}^m y^2_i & \sum_{i=1}^m y_i \cr \sum_{i=1}^m x_i & \sum_{i=1}^m y_i & \sum_{i=1}^m 1 \cr } \right] \left[ \matrix {A \cr B \cr C \cr} \right] = \left[ \matrix {\sum_{i=1}^m x_iz_i \cr \sum_{i=1}^m y_iz_i \cr \sum_{i=1}^m z_i \cr} \right] $$

I would like to extend this to calculate my required equation. What are the matrix values that I need?

UPDATE

By "reverse engineering" two examples given in the PDF, I came up with this answer: $$ \left[ \matrix{ \sum_{i=1}^m x_i & \sum_{i=1}^m x^2_i & \sum_{i=1}^m x_iy_i & \sum_{i=1}^m x^3_i & \sum_{i=1}^m x_iy^2_i & \sum_{i=1}^m x^2_iy_i \cr \sum_{i=1}^m y_i & \sum_{i=1}^m x_iy_i & \sum_{i=1}^m y^2_i & \sum_{i=1}^m x^2_iy_i & \sum_{i=1}^m y^3_i & \sum_{i=1}^m x_iy^2_i \cr \sum_{i=1}^m 1 & \sum_{i=1}^m x_i & \sum_{i=1}^m y_i & \sum_{i=1}^m x^2_i & \sum_{i=1}^m y^2_i & \sum_{i=1}^m x_iy_i \cr } \right] \left[ \matrix {A \cr B \cr C \cr D \cr E \cr F \cr} \right] = \left[ \matrix {\sum_{i=1}^m x_iz_i \cr \sum_{i=1}^m y_iz_i \cr \sum_{i=1}^m z_i \cr} \right] $$ The "order" of the terms in the matrix are not in the same pattern as in the original matrix above, but I wanted to preserve the $flat$ equation order.

Is it correct?

Answer 1

I don't agree with your answer.

Answer 2

We are given a list of pixel locations $(x_i, y_i)$, for $i = 1, \ldots, m$, and corresponding pixel intensity values $z_i$. Our goal is to find numbers $A, B, C, D, E$, and $F$ such that $$ z_i \approx A + B x_i + C y_i + D x_i^2 + E y_i^2 + F x_i y_i $$ for $i = 1, \ldots, m$. These equations can be written concisely using matrix notation as $$ X \beta \approx z, $$ where $$ X = \begin{bmatrix} 1 & x_1 & y_1 & x_1^2 & y_1^2 & x_1 y_1 \\ \vdots & & & & & \vdots \\ 1 & x_m & y_m & x_m^2 & y_m^2 & x_m y_m\\ \end{bmatrix}, \qquad \beta = \begin{bmatrix} A \\ B \\ C \\ D \\ E \\ F \end{bmatrix}, \quad \text{and} \quad z = \begin{bmatrix} z_1 \\ \vdots \\ z_m \end{bmatrix}. $$ We choose the vector $\beta$ to minimize $L(\beta) = \frac12 \| X \beta - z \|_2^2$. This is a standard least squares problem, and we can solve it by setting the gradient of $L$ equal to $0$, which yields the so-called "normal equations": $$ \nabla L(\beta) = X^T(X \beta - z) = 0 \implies X^T X \beta = X^T z. $$ Note that $X^T X$ is a $6 \times 6$ matrix. We can solve this linear system of equations to obtain $\beta$.