A statistical robust least-squares problem

65 Views Asked by At

Consider the following statistical robust least-squares problem

$$\text{minimize } \mathbb E \|Ax - b\|_2^2$$

where $A = \bar A + U$, where $\bar A$ is the mean of $A$ and $U$ is a zero-mean random matrix. We can express the objective as $$\mathbb E \|Ax - b\|_2^2 = (\bar Ax-b)^T(\bar Ax - b) + \mathbb E [x^TU^Tux]$$

Why can we can express the objective as $E ||Ax - b||_2^2 = (\bar Ax-b)^T(\bar Ax - b) + E[x^TU^TUx]$

Isn't $E ||Ax - b||_2^2 = E(\bar Ax-b)^T(\bar Ax - b) + 2E(\bar A x - b) + E[x^TU^TUx]$?

1

There are 1 best solutions below

4
On

Note that \begin{align*} & \mathbb E \| Ax - b \|_2^2 \\ =& \mathbb E \left[ [(\bar A x - b )^T + (U x)^T][(\bar Ax - b) + (Ux)] \right] \\ =& \mathbb E[(\bar Ax - b)^T(\bar A x - b)] + \mathbb E[(\bar Ax - b)^T(Ux)] + \mathbb E[(\bar Ax - b) (Ux)^T] + \mathbb E[(Ux)^T(Ux)] \\ =& (\bar A x - b)^T (\bar A x -b) + (\bar A x - b)^T \mathbb E[Ux] + (\bar A x - b) \mathbb E[(Ux)^T] + \mathbb E[x^T U^T U x] \end{align*} where the last step is justified because $\bar A$, $b$, and $x$ are deterministic and behave like constants with respect to expectations. Since $U$ has mean $0$, the middle terms vanish.

The issues with your alternative formulation are (1) we don't need an expectation on the leftmost term because it's not random, and (2) the $Ux$ terms seem to have been dropped from the middle components.