why the name "relaxation" in the SOR Gauss-Seidel iterative method?

Question

The question is that simple. It is in the subject.

Why the name "relaxation" in the SOR Gauss-Seidel iterative method?

I have googled for it without luck.

Moreover, when $\omega$ is used in Gauss-Seidel is named "relaxation" and when $\omega$ is used in Jacboi is named: "weight" weighted Jacobi method

So, something got to be special about Gauss-Seidel to deserve such a name.

I observed that there is a "relaxation" technique explained in this thesis back from 1948. I do not see a connection between relaxation in that context of mechanics (PDEs) and the relaxation asked here which is in the context of linear systems such as Jacobi and Gauss-Seidel. Hence, I do not consider this as a duplicate questions.

Thanks.

Answer 1

Being curious about the same question, I read the related part of the thesis:

Southwell considered a framework having elastic members and frictionless joints. Rigid constraints were imposed at the joints such that the movements of the joints were prevented but the ends of the members were left free to turn. All joints were constrined in this manner and specified forces applied. Since the constraints were rigid they sustrained the whole force at every joint.

Then one constrint was relaxed so that one joint could move slowly (so that equilibrium is maintaned and vibrations are not excited) through a specified distance in a specified direction. If the initial force on the constrint had a component in the direction of travel, this component of the force was relieved and the strain energy stored in the member. All joints but one having remained fixed, it was possible to calculate how much force was transfered.

Another constraint was relaxed and a similar operation performed. Systematically proceedings whith each constraint the amount of total strain energy was increased with each step until the forces remaining on the constraints were so small as to be negligible. Since each constraint was "relaxed" in turn, the name "Relaxation Methods" was given to the procedure.

So for iterative method in solving linear equation system $Ax=b, (Ax^k-b=r^k)$, my understanding is that:

• Each component of the unknown (i.e., $x_i$) is regarded as a joint constraint;
• Each component of the residual (i.e., $r_i$) is regarded as a force on the corresponding joint;
• Relaxing "joint" $x_i$ means $x_i$ is allowed to be updated, in order to diminish the corresponding "force" $r_i$. This is possible since only $x_i$ is relaxed while all the rest are still fixed (All joints but one having remained fixed, it was possible to calculate how much force was transferred).
• Other $x_i$ are relaxed in turn similarly, until the residual $r_i$ on all joints are so small to be negligible.

So far the analogy above is satisfactory to me, although I don't know the details of the mechanics/physics involved.