Consider the system \begin{align}a=b-2\\ b=2a+14\end{align} Implementing a simple Jacobi method and initializing at $(a,b)=(0,0)$, updating by $a^{k+1}=b^{k}-2$, $b^{k+1}=2a^k+14$, we get: $(-2,14),(12,10),(8,38),(36,30),(28,86)\cdots$, and the resulting sequence does not converge.
By contrast, consider the equivalent description \begin{align}a=\frac b2 - 7\\ b=a+2\end{align} and the corresponding update rules $a^{k+1}=\frac {b^k} 2-7$, $b^{k+1}=a^k+2$. Here, the Jacobi method quickly converges to the truth, $a=-12$, $b=-10$.
What's with the difference between the two? Why does Jacobi work on the second version and not on the equivalent first version? I get that the matrix describing this system is not strictly diagonally dominant, and that therefore Jacobi is not guaranteed to converge, but I still find it surprising that it converges for only one of the above two statements of the problem.
The matrix that defines the "original" linear equations is $$A_1 = \begin{pmatrix}1 & -1 \\ -2 & 1 \end{pmatrix}$$ for unknowns $\begin{pmatrix} a \\ b \end{pmatrix} $ and right-hand-side $b_1 = \begin{pmatrix} -2 \\ 14 \end{pmatrix} $. The error propagation matrix $C$ such that $$e^{k+1} = C e^k$$ for the Jacobi method is given by $$C = I - D^{-1} A \Rightarrow C_1 = I - IA_1 = \begin{pmatrix}0 & 1 \\ 2 & 0 \end{pmatrix}$$ where $D$ is the diagonal matrix consisting of the diagonal elements of $A$. The spectral radius of $C_1$ is $\rho(C_1) = \sqrt{2} > 1$, thus convergence is not guaranteed (as observed). Now consider the second formulation with $$A_2 = \begin{pmatrix}1 & -0.5 \\ -1 & 1 \end{pmatrix}$$ with right-hand-side $b_2 = \begin{pmatrix} -7 \\ 2 \end{pmatrix} $. Here, $$C_2 = \begin{pmatrix}0 & 0.5 \\ 1 & 0 \end{pmatrix} $$ and the spectral radius is $\rho(C_2) = 1/\sqrt{2} < 1$ thus convergence is guaranteed for every starting vector.