Q:
I saw Yousef Saad said "The Jacobi iteration determines the i-th component of the next approximation so as to annihilate the i-th component of the residual vector" in his book <<Iterative Methods for SAparse Linear System>>. But actually, it's not. Take the example in this notes, we could simply find $(b-Ax_1)_1$, which is the first element of the residual produced after the first iteration, is not $0$, which means we did not annihilate the $1th$ component of the residual vector after the first iteration. So my question is, what does Yousef Saad mean here?
What I found:
I saw other articles that described Jacobi iteration derive the Jacobi Method by simply using Linear Algebra to get iterative formula: $$ x_{(k+1)} = D^{-1}(E+F)x_k + D^{-1}b $$ where $x_{k+1}$ is the $(k+1)th$ solution and $A=D-E-F$.
Yousef try to derive this formula by giving us what Jacobi iteration does in each iteration just like what I mentioned above, he said: $$(b-Ax_{k+1})_i = 0$$ which means after $k+1$ iteration, then corresponding $i-th$ residual element of current solution $x_{k+1}$ is $0$. (Actually I can't understand this, I mean, what is the relation between $k+1$ and $i$). Then Yousef derive this formula: $$ a_{ii} x_i^{(k+1)} = - \sum _{j=1,j\neq i}^n a_{ij}x_j^k + b_i$$ I can't either understand this, how does Yousef link $x^{k+1}$ and $x^{k}$? Then he sum up these component-wise form to get the final formula: $$ x_{(k+1)} = D^{-1}(E+F)x_k + D^{-1}b $$.
I get the point of Yousef in this sector, But what he said really confused me at the beginning, Yousef really needs to add "TEMPORARILY" in his book...
Original: "The Jacobi iteration determines the $ith$ component of the next approximation so as to annihilate the $ith$ component of the residual vector"
Response: this is true but not entirely true since this method doesn't annihilate the $ith$ residual component forever. The residual component we discarded would be introduced again when we approximate the next solution element of the current iteration.
Take the first two beginning steps as an example, Initiative solution is $x_0$ (NOTATION: $x_i^j$ is the $ith$ component after $jth$ iteration). So the first equation of linear system is $$a_{11}x_1^0 + a_{12}x_2^0 + \dots + a_{1n}x_n^0 = \hat{b_1}$$
where $\hat{b_1}$ is the value of left-hand side formula.
Now we rectify $x_1^0$ to $x_1^1$(the element of $x^1$ after the first iteration) so that the left-hand side formula equals $b_1$.
Taking $(x_1^1, x_2^0, \dots, x_n^0)$ as a temporary solution we "annihilate the $i-th$ component of the residual vector" TEMPORARILY since we would introduce it again when we update $x_2^1$. So after the first iteration completely (which means we get the current answer $(x_1^1, x_2^1, \dots, x_n^1)$). Not a single residual component was eliminated.
As a result, we CAN'T use this derivative process as an intuitive understanding at all, it's useless in my view...