I have a question that says use a relative tolerance of $10^{-3}$ and asks if the estimate errors are in line with the actual errors.
What does relative tolerance mean and how do you work out estimate errors?
Assuming actual error is the difference between the value you get by iteration compared to the value it is converging too?
Thank you.
edit: picture added.
We want to implement the Jacobi Iteration Algorithm to solve the system $Ax = b$, where:
$$ A = \begin{bmatrix} 3 & -1 & 1\\ 3 & 6 & 2 \\ 3 & 3 & 7\end{bmatrix}, b = \begin{bmatrix}1 \\ 0 \\ 4 \end{bmatrix}$$
The goal is a relative tolerance (input tolerance to the algorithm) of $10^{-3}$ with a starting vector of zero. This means we are expecting three digits of accuracy in the result.
The intermediate iterations with these conditions follow (note the problem only asked to display three digits).
$$ \begin{array}{c|ccc} n & x_1 & x_2 & x_3 \\ \hline 0 & 0 & 0 & 0 \\ 1 & 0.3333330 & -0.000000 & ~0.571429 \\ 2 & 0.1428570 & -0.357143 & ~0.428571 \\ 3 & 0.0714286 & -0.214286 & ~0.663265 \\ 4 & 0.0408163 & -0.256803 & ~0.632653 \\ 5 & 0.0368481 & -0.231293 & ~0.663994 \\ 6 & 0.0349044 & -0.239755 & ~0.654762 \\ 7 & 0.0351609 & -0.235706 & ~0.659222 \\ 8 & 0.0350240 & -0.237321 & ~0.657377 \\ 9 & 0.0351008 & -0.236638 & ~0.658127 \\ 10 & 0.0350784 & -0.236926 & ~0.657801 \end{array} $$
As you can see, the iterations converged in ten steps to:
$$\begin{bmatrix}x_1 \\ x_2 \\ x_3 \end{bmatrix} = \begin{bmatrix}~0.0350784 \\ -0.236926 \\ ~0.657801 \end{bmatrix}$$
Calculating the actual results yields:
$$\begin{bmatrix}x_1 \\ x_2 \\ x_3 \end{bmatrix} = \begin{bmatrix} \dfrac{2}{57} \\ -\dfrac{9}{38} \\ \dfrac{25}{38} \end{bmatrix} \approx \begin{bmatrix} ~0.0350877 \\ -0.236842 \\ ~0.657895 \end{bmatrix}$$
As you can see, the algorithm results are within $10^{-3}$ (three digit accuracy). As a comparison, if we wanted $6-$digits of accuracy, that is, a relative tolerance of $10^{-6}$, we need $17$ iterations.
When we solved for $x_i$ in terms of $x_1, x_2,..., x_{i-1}, x_{i+1},...,x_n$, as part of the Jacobi Iteration algorithm, we already had the equations in a form with a leading largest coefficient $a_{ii}$. This will lead to a solution when the system is consistent.
If we did not have that, the algorithm may pass or fail. See the Jacobi Iteration convergence discussion.