I'm trying to solve the over-determined system of linear form:
$$ Ax = b $$
I know we can consider the problem as a least-square regression[1]:
$$ min_x \|Ax - b\| $$
and the solution is
$$ x = (A^\top A)^{-1}A^\top b $$
But I don't know how accurate this approximation is. How to properly evaluate the uncertainty of x of this solution?
Also appreciate if you share some article about this theory.
Note:
I found related answer[2]. According to the article, the covariance matrix of x is calculated as
$$ C_x = (A^\top A)^{-1}A^\top C_b A (A^\top A)^{-1} $$
If it is true, then, how can we set covariance matrix of b ($C_b$)? Just calculating standard deviation of $\|Ax - b\|$ (we denote this as $\sigma$) and set $C_b = eye(\sigma^2)$ is O.K.?