Finding the design matrix

801 Views Asked by At

In order to approximate the mean sea level over different places $A,B$ and $C$, six measurements have been made:

$h_A=12, \ h_B=23, \ h_C=35, \ h_B-h_A=16, \ h_C-h_B=10, \ h_C-h_A=20.$

The system of equations for the heights becomes

$$\begin{pmatrix} 1 & 0 & 0 \\ 0 &1 & 0 \\ 0 & 0 & 1 \\ -1 & 1 & 0 \\ 0 & -1 & 1 \\ -1 & 0 & 1 \end{pmatrix}\begin{pmatrix}h_A \\h_B \\ h_C \end{pmatrix}=\begin{pmatrix}12 \\23 \\ 35 \\ 16 \\ 10 \\ 20\end{pmatrix}.$$

This equation is of the form $Xh=y$.

Questions:

1) If $h_A=12$ and $h_B=23,$ how can $h_B-h_A=16?$ I don't understand those three last measurements. Why having the differences and not just proceeding with $h_D,h_E$ and $h_F?$

2) I find it really hard to construct the design matrix $X$ from the given measurements. My book is terrible at explaining this. Can anyone briefly outline how this is done generally?

1

There are 1 best solutions below

2
On BEST ANSWER

2) You have constructed the design matrix correctly: for each of the measurement you are given, you add a row to the design matrix and the row is filled with the coefficients multiplying your unknown model parameters.

1) As you observed, there is inconsistance between the observations. This means that only have access to noisy observations, i.e. you actually have $$ y = Xh + \epsilon $$ where $\epsilon$ is an unknown noise vector of size $4$. A possible way to solve this ill-posed problem is to do a least square linear regression (it corresponds to making the assumption that the noise is typical and follows a normal distribution). Your aim is to recover the most likely $h$ given your noisy observations, which is the vector $h$ that minimizes the "strength" (here the squared euclidean norm) of the estimated noise. This reads $$ \min_h \frac{1}{2}\Vert Xh- y\Vert^2_2. $$ This is a convex function to minimize so it is sufficient to find the point where the derivative vanishes to find the minimizer. We find $h^*=(X^TX)^{-1}X^Ty$ which is the desired estimator for your mean see level (here $X^T$ is the transposed matrix of $X$).

If you have not more information on the problem then it is reasonable to use least-square regression, but in case the noise is not "normal" then other approaches should be considered.