Least squares problem: find the line through the origin in $\mathbb{R}^{3}$

Question

The problem is as follows:

"Please set up (but do not solve) the normal equations for the following least squares approximation problem: Find $(a, b, c, d)$ such that the plane H described by $ax + by + cz = d$ minimizes $\sum |ax_i + by_i + cz_i − d|^2$ where the $(x1, y1, z1), \cdots ,(x6, y6, z6)$ are the following points: $(2,3,4)$, $(99,−85,0)$, $(0,1,8)$, $(5,2,2)$, $(3,3,3)$, $(1,2,4)$."

I solved a similar problem with points representing ordered pairs, for example $(2,1)$, $(3,5)$ and so on, but the question was to get a line that approximates such ordered pairs. In that case I represented the points as $y = mx + b$ and got $m$ and $b$ for the new line. I wonder if for the new problem which is a 3D space I also have to represent the ordered triplets as $z = ax + by + c$ or something like that.

I will very much appreciate any clue.

Answer 1

You are on the right track. Set up your equations like you would in 2 dimensions. \begin{align*} z_1 &= d + ax_1 + by_1 \\ z_2 &= d + ax_2 + by_2 \\ &\vdots \\ z_6 &= d + ax_6 + by_6. \end{align*}

From this, you can hopefully derive the normal equations as matrix products.

Answer 2

We have our data points $\left\{ x_{k}, y_{k}, z_{k}, d_{k} \right\}_{k=1}^{m}$, and our trial function, a line in $\mathbb{R}^{3}$ through the origin: $$ d\left( x, y, z \right) = a x + b y + c z. $$

The linear system is $$ \begin{align} \mathbf{A} a &= d \\ % \left[ \begin{array}{ccc} x & y & z \end{array} \right] % \left[ \begin{array}{c} a \end{array} \right] % & = % \left[ \begin{array}{c} d \end{array} \right]\\[3pt] % \left[ \begin{array}{ccc} x_{1} & y_{1} & z_{1} \\ x_{2} & y_{2} & z_{2} \\ \vdots & \vdots & \vdots \\ x_{m} & y_{m} & z_{m} \\ \end{array} \right] % \left[ \begin{array}{c} a \\ b \\ c \end{array} \right] % &= % \left[ \begin{array}{c} d_{1} \\ d_{2} \\ \vdots \\ d_{m} \end{array} \right]. % \end{align} $$

The normal equations are $$ \begin{align} \mathbf{A}^{*} \mathbf{A} a &= \mathbf{A}^{*} d \\ \left[ \begin{array}{ccc} x \cdot x & x \cdot y & z \cdot z \\ y \cdot x & y \cdot y & y \cdot z \\ x \cdot z & x \cdot z & z \cdot z \\ \end{array} \right] % \left[ \begin{array}{c} a \\ b \\ c \end{array} \right] % &= % \left[ \begin{array}{c} x \cdot d \\ y \cdot d \\ z \cdot d \end{array} \right]. % \end{align} % $$ The solution is $$ a_{LS} = % \left[ \begin{array}{c} a \\ b \\ c \end{array} \right]_{LS} % = \left( \mathbf{A}^{*}\mathbf{A} \right)^{-1} \mathbf{A}^{*} d. $$