Sorry if anything is wrong or missing, since this is my very first post in this forum! Anyway, I've been stuck for some days with a statistical, control theory oriented question, which I will state now:
Question: Consider the system $y = H\theta + e$, where, $$ y = [y_1; y_2]^T, \quad H=[H_1;H_2]^T, \quad e=[e_1; 0]^T$$ and $E(e_1)=0$, $E(e_1e_1^T)=I$. Find the estimates of $\theta$ and $y_2$ given $y_1, H_1$ and $H_2$. Of course, the semi-colons are just used to indicate matrix concatenation. Also, you can use this relation if needed: $$ \begin{bmatrix} \mathbf{A} & \mathbf{B} \\ \mathbf{C} & \mathbf{D} \end{bmatrix}^{-1} = \begin{bmatrix} \mathbf{A}^{-1}+\mathbf{A}^{-1}\mathbf{B}(\mathbf{D}-\mathbf{CA}^{-1}\mathbf{B})^{-1}\mathbf{CA}^{-1} & -\mathbf{A}^{-1}\mathbf{B}(\mathbf{D}-\mathbf{CA}^{-1}\mathbf{B})^{-1} \\ -(\mathbf{D}-\mathbf{CA}^{-1}\mathbf{B})^{-1}\mathbf{CA}^{-1} & (\mathbf{D}-\mathbf{CA}^{-1}\mathbf{B})^{-1} \end{bmatrix} $$ where all dimensions need to match accordingly.
First thing I thought just by gazing at the question was: "use the first line of the parametric system for a least squares fit and then use the gained $\theta$ to get $y_2$!". Although it may be possible to do it like this, I strongly feel that information was not used as needed, since the variance of the error $e_1$ was not used in any way as much as the given inversion relation.
On my second try, I thought to experiment on another least squares approach, by directly using the pseudo-inverse in $H$, then expanding it using the relation, and finally manipulating everything to isolate something about $\theta$ and $y_2$. Frankly, I did not go very far on this method.
Lastly, I tried to minimise some expectation about $y$ to find a possible recurrent relation of some sorts, $$E((\hat y - y)^T(\hat y - y))$$ where $\hat y$ was the estimative $\hat y = H \hat \theta$. Needless to say, no good results were found by me that could even use the information given in the question.
So, my real question is: could you find a way to solve this estimator deduction, using not very advanced probability and statistical techniques (I would say that everything used on kalman filter and LQG are the ideal), while also using all the hints and hypothesis given in the question?
Really appreciate any attempt at this one, and sorry if my english gets sloppy any time along the text! best wishes.
Edit with compelling answer:
You can consider the system in its matrix form:
$$ \left[ \begin{array}{c} y_1 \\ y_2 \end{array} \right] = \left[ \begin{array}{c} H_1 \\ H_2 \end{array} \right] \theta + \left[ \begin{array}{c} e_1 \\ 0 \end{array} \right] $$
and manipulate it so that the $y_2$ ends up as another parameter in this system,
$$ \left[ \begin{array}{c} y_1 \\ 0 \end{array} \right] = \left[ \begin{array}{cc} H_1 & 0 \\ -H_2 & 1 \end{array} \right] \left[ \begin{array}{c} \theta \\ y_2 \end{array} \right] + \left[ \begin{array}{c} e_1 \\ 0 \end{array} \right] $$
From here, you can just use a normal least squares method, the given inverse relation and obtain estimates for both $\theta$ and $y_2$.