Let us consider a linear system of equations of the form: $$ Ax = b, $$ where $A \in \mathbb{R}^{m\times n}$ and $b \in \mathbb{R}^m$. Such a linear system naturally arises in many fields of applications. In particular, if $m < n$ and $A$ is full-rank, the system allows infinitely many solutions.
Often, among infinitely many solutions, one seeks to find the minimum norm solution. That is, $$ \min_x \|x\| \quad \text{subject to} \quad Ax=b. $$ If $\|\cdot\|$ is the standard Euclidean norm, the minimum norm solution is $A^{\dagger}b$, where $A^{\dagger}$ is the Moore-Penrose inverse of $A$.
I am wondering if there is any convincing reason why one should look for the minimum norm solution in general. I understand that the nature of the problem allows infinitely many solutions and one needs to select one of them (to solve the system). One possible reason I can think of is that the minimum norm solution might be easier to solve numerically (stable and robust). However, I cannot think of any other reasons.
Also, it is easy to see from many different fields (especially along with the optimization problem), people use regularization terms in order to control the norm of the variables of interests. For example, $$ \min_x F(x), \quad F(x) = \|Ax- b\|^2 + \lambda \|x\|^2, $$ which always has a unique solution for any $\lambda > 0$. This is also somehow looking for the minimum norm solution. I think there is some universality in this approach, however, I cannot see it at the moment.
Any insights, answers, or comments will be very appreciated. Thanks!