I would like some suggestions to tackle the following problem.
Given a system
$y = X\beta$
where $y \in \mathcal{R}^m$, $X \in \mathcal{R}^{m \times n}$ and $\beta \in \mathcal{R}^n$,
I want to estimate $\beta$.
However, I have perturbations in both $y$ and $X$, i.e. I am given
$y+\delta y = (X+\delta X)\hat{\beta}$.
What is the right way to estimate $\hat{\beta}$ such that the latter is as close as possible to $\beta$?
Why not approaching it the usual way? I.e., without "perturbation" you have $$ y = X\beta + \epsilon, $$
hence, $$ \epsilon= y - X\beta. $$ With "perturbation" you have $$ y + \delta y = (X + \delta X)\beta = X\beta + \delta X\beta, $$ thus $$ y=X\beta + \delta(X\beta-y)=X\beta - \delta\epsilon=X\beta+\tilde{\epsilon}. $$ And then proceed with the usual technique to estimate $\beta$?