What's the most efficient way to fit a surface to three or more points?

Question

Say I have a function of the form $s=b-mp+at$, where $p$ and $t$ are the independent variables, and I have 3 or more points of the form $(p,t,s)$. I want to find the best values for $b$, $m$, and $a$ to give me the best fit. Is there an analogue of linear regression for this situation?

Answer 1

Here are some matrix algebra equations you can try. I'm suggesting this approach because you mentioned that there may be future cases where you have more than three points to work with and in such cases the linear regression solution is what you want.

Starting with your equation for $s$, define the following quantities:

$$\underline s = \begin{bmatrix} s_1\\ s_2\\ s_3 \end{bmatrix}$$

$$X = \begin{bmatrix} 1 \ \ p_1 \ \ t_1\\ 1 \ \ p_2 \ \ t_2\\ 1 \ \ p_3 \ \ t_3\\ \end{bmatrix}$$

$$\underline\beta = \begin{bmatrix} b\\ m\\ a \end{bmatrix}$$

Then per your equation $\underline s = X \ \underline\beta$. In your case you have three data points and 3 unknowns. If $X$ is full column rank (its determinant is not zero, points are not colinear) and since $X$ is square, multiplication of both sides of the equation for $ \underline s$ by $X^{-1}$ gives $\underline\beta = X^{-1} \underline s$.

If you have more than three data points (more than 3 rows of the $s$ vector and the X matrix) the following equation will do the trick: $$\underline\beta = (X'X)^{-1}X' \underline s$$ again as long as $X$ is full column rank, or equivalently, as long as $det(X'X) \neq 0$. In this case the equation gives the least squares estimate for $\underline \beta$.

The second equation will work as well for the case of only 3 points, which means you can solve your initial problem with a standard regression package. In this case you have a saturated model which has no degrees of freedom left over to estimate an error term, but with three points and three parameters to estimate you'll have a perfect fit, as has been pointed out in the comments. I hope this helps.