least squares approximation method for computing the inverse of the matrix $A$

1.2k Views Asked by At

Compute the least squares approximate inverse of matrix $A$,

$A=\pmatrix{2&-1\\ -1&2&-1&\\&\ddots&\ddots&\ddots\\ &&-1&2&-1\\ &&&-1&2}_{7\times7}$

using the sparse pattern $S=pattern(A)$.

Here is my doubt, I think A is full rank, so we can compute its inverse directly, but why this problem need to use the least squares approximation method to compute its inverse, how is it written in matlab code? Sorry for me not doing preview before asking this question, because this course is called numerical linear algebra and I did not take the course numerical analysis before. Really appreciate for someone explain for me, many thanks.

1

There are 1 best solutions below

0
On BEST ANSWER

You are confusing two different terms.

The goal is not to compute the inverse matrix of $\mathbf A$, rather you should seek a matrix $\mathbf M$ which is an approximation of $\mathbf A^{-1}$. The goal is to find $\mathbf M$ such that $\mathbf A \mathbf M \approx \mathbf I$, where $\mathbf I$ is the identity matrix. If you want to solve $$\mathbf A \mathbf x= \mathbf b,$$ then the equivalent linear system $$\mathbf A \mathbf M \mathbf y=\mathbf b$$ can often be solved quickly with respect to $y$ using an iterative solver. The approximate sparse inverse $\mathbf M$ will be computed by solving a sequence of least squares problems.

Typically, we seek the matrix $\mathbf M$ such that $\|\mathbf A \mathbf M - \mathbf I\|_F$ is minimized over all matrices $\mathbf M$ with a specific sparsity pattern. Here $F$ refers to the Frobenious norm of the matrix. There are sound mathematical reasons for using the sparsity pattern of $\mathbf A$, but they are not important right now. The Frobenius norm has the advantage that the minimization problem decouples, specifically $$ \|\mathbf A \mathbf M - \mathbf I\|_F^2 = \sum_{j=1}^n \|(\mathbf A \mathbf M- \mathbf I)e_j\|_2^2 = \sum_{j=1}^n \|(\mathbf A \mathbf m_j-e_j\|_2^2.$$ Here $\mathbf e_j$ is the $j$th column of the identity matrix and $\mathbf m_j$ is the $j$th column of the matrix $\mathbf M$. The left hand side can be minimized, by minimizing the individual terms on the right side.

Now, given the sparsity pattern of $\mathbf M$, you immediately know which entries of $\mathbf m_j$ must be zero. The remaining entries are your unknowns. Ex. for $j=2$, there are three possible nonzeros $m_{i2}$ where $i=1,2,3$. Your task is minimize $$\|\mathbf A \mathbf m_2 - \mathbf e_2\|_2.$$ You do this by solving in the least squares sense the linear system $$ \begin{bmatrix} \mathbf a_1 & \mathbf a_2 & \mathbf a_3 \end{bmatrix} \begin{bmatrix} m_{12} \\ m_{22} \\ m_{32} \end{bmatrix} = \mathbf e_2.$$ Here $\mathbf a_i$ refers to the $i$th column of $\mathbf A$. This system is overdetermined with 7 equations in 3 unknowns. You gets similar linear least squares systems for the other columns of $\mathbf M$.


If you are using MATLAB, then know that the backslash operator is overloaded and returns the linear least squares solution for overdetermined linear system.