I know that for a square non-singular matrix right and left inverse are the same. For $n \times m$ matrix $A$ both inverses have size $m \times n$ and:
- $A'(AA')^{-1}$ is the right inverse
- $(A'A)^{-1}A'$ is the left inverse
Now I can subtract right inverse from left and calculate the norm. I am wondering whether this number has any particular meaning?
To Velut luna. Here is an example that left and right inverse both exist and they are different (python code):
import numpy as np
from numpy.linalg import inv
from numpy.linalg import norm
A = np.random.randint(-5, 5, size=(3, 5))
A1 = A.T.dot(inv((A.dot(A.T))))
A2 = inv(A.T.dot(A)).dot(A.T)
print A
print
print A1
print
print A2
You can try it here (just click New->python2 and paste this code).
As far as I understand, for non-square matrices, the left and right inverse cannot both exist.
In fact it is easy to show that should they both exist, then they had to be the same:
Consider an $m \times n$ matrix $A$. If $n \times m$ matrix $B$ is the right inverse of $A$, viz., $AB = I_m$. Now if given that $A$ also has a left inverse $C$, which is an $n \times m$ matrix such that $CA=I_n$. Then it can be shown that $B=C$ because
$$C=CI_m=C(AB)=(CA)B=I_nB=B.$$
In your formula, (I suppose $A'$ means $A^T$) you use the fact that $AA^T$ and $A^TA$ are both invertible. But $A^TA$ is invertible only when $A$ has independent columns, and $AA^T$ is ivertible only when $A^T$ has independent columns. However, for non-square matrices, $A$ and $A^T$ cannot both have independent columns.