Least-squares solution to a linear matrix equation

558 Views Asked by At

Let $\\A$ be a matrix of size $\\(m, n)$, $\\b$ a column vector of size $\\m$, $\\x$ a column vector of size $\\n$ and $\\a$ a real number.

If $\begin{bmatrix} x \\ a \end{bmatrix}$ is the least-squares solution to the linear matrix equation:

$$ \begin{bmatrix} A & \begin{matrix} -1 \\ -1 \\ ... \\ -1 \end{matrix} \end{bmatrix} \begin{bmatrix} x \\ a \end{bmatrix} = b $$

does it mean that the arithmetic mean of $\\A x - b$ equals $\\a$?

As background information, I used that result to answer this post on stackoverflow.

1

There are 1 best solutions below

0
On BEST ANSWER

Yes, this is actually a restatement of the following well-known fact: if the design matrix contains a column of all ones (or equivalently, all negative-ones), then the sum of the residuals is zero. For a short proof, let $e$ denote the column of all ones, $X = [A, -e]$, by the nature of least squares, the residual vector $b - (Ax - ae)$ is perpendicular to the linear space spanned by the columns of $X$. In particular, it is perpendicular to the vector $e$, thus, $$0 = e^T(b - Ax + ae) = an - e^T(Ax - b)$$ or $a = e^T(Ax - b)/n$, which is your assertion.