Fairly basic question:
Given the relation d = Dm and the quantities d and a sparse D I can find a solution m = (D'D)^(-1)D'd by subbing the known quantities in the equation for m. However some code that I am studying opts to use an iterative approach to find m. Is there any benefit in doing this?
On
The only advantage of the iterative method is that you need not recalculate everything again if points are added. But if you have all points anyway, your approach is equally efficient.
On
In addition, sometimes, the problem dimensions are so large that an explicit inversion of $D^TD$ is just not feasible. In this case, we sometimes resort to iterative methods to approximate the LS solution which get away with only using matrix vector products of the form $D m$ and $D^T p$. These can be implemented very efficiently for lage sparse matrices.
If your problem is so small that you can just compute the LS in one step, there is no big need to worry about these.
$\bf D$ is sparse.
However ${\bf D}^{-1}$ or even $({\bf D}^T{\bf D})^{-1}$ is not necessarily sparse and can require lots of memory and computational resources to compute.
By using an iterative method that only use matrix-vector multiplications (and no matrix-matrix operations, like many other solvers do), we can be sure that any operation will be very fast, since multiplication by $\bf D$ on any matrix will be fast thanks to the sparsity.
Another benefit is that an iterative method for many systems you can get an acceptable approximation to a solution a lot faster than an exact solution. Depending on your application this can be a very valuable tradeoff.