To give some context, I'm trying to solve the following problem:
$y = BA^{-1}x$
where:
$y$ = $n \times 1$ vector -- is known
$x$ = $3 \times 1$ vector -- is unknown
$B$ = $3 \times n$ matrix -- is known
$A$ = $n \times n$ singular matrix -- So $A$ is known and cannot be inverted to solve the problem.
If I could compute a generalized inverse for $B$ (let's call it $B^\dagger$) than I could avoid dealing with $A^{-1}$ altogether. The solution would simply be:
$x = AB^\dagger y$
However for this to work $B^\dagger$ would have to be a left inverse, i.e. $B^\dagger B = I$.
Any ideas on a generalized inverse method that would produce a left inverse for a short (less rows than columns) matrix? Is this even possible?
Forgive my confusion. The question states $\mathbf{A}$ is singular (a classic inverse does not exist), yet the formula uses $\mathbf{A}^{-1}$ (the classic inverse). The answers are for different assumptions.
If the matrix product $\mathbf{C}=\mathbf{B}\,\mathbf{A}$, is known, build the psuedoinverse from the singular value decomposition: $$ \mathbf{C} = \mathbf{U}\, \Sigma\, \mathbf{V}^{*} \qquad \Rightarrow \qquad \mathbf{C}^{\dagger} = \mathbf{V}\, \Sigma^{\dagger} \,\mathbf{U}^{*} $$
Both matrices $\mathbf{A}$ and $\mathbf{B}$ will also have a pseudoinverse. General comments follow.
Classification of inverses
Start $\mathbf{A}\in\mathbb{C}^{m\times m}_{m}$. The matrix is square and has full rank. Therefore the classic inverse $\mathbf{A}^{-1}$ exists and satisfies two equalities: $$ \mathbf{A}^{-1}\mathbf{A} = \mathbf{A}\mathbf{A}^{-1} = \mathbf{I}_{m} $$ The classic inverse is both a left and a right inverse $$ \mathbf{A}^{-L}\mathbf{A} = \mathbf{A}\mathbf{A}^{-R} = \mathbf{I}_{m} $$
What if we can only satisfy one of these inequalities? By relaxing the requirement, we generalize the notion of an inverse. The least squares problem naturally develops the Moore-Penrose pseudoinverse, $\mathbf{A}^{+}$.
If $\mathbf{A}^{+} \mathbf{A}= \mathbf{I}_{n}$, then the pseudoinverse $\mathbf{A}^{+}$ is a left inverse. When $\mathbf{A} \mathbf{A}^{+}= \mathbf{I}_{n}$, the pseudoinverse $\mathbf{A}^{+}$ is a right inverse. When the pseuodinverse is a left and right inverse, the pseudoinverse in the classic inverse.
The remaining case is when the pseudoinverse is neither a left nor a right inverse. There is no identity inequality. The pseudoinverse is a projector on the column space of $\mathbf{A}$.
Watch the null spaces
Row and column rank deficient
For $\mathbf{A}\in\mathbb{C}^{m\times n}_{\rho}$, the singular value decomposition is $$ \begin{align} \mathbf{A} &= \mathbf{U} \, \Sigma \, \mathbf{V}^{*} \\ % &= % U \left[ \begin{array}{cc} \color{blue}{\mathbf{U}_{\mathcal{R}}} & \color{red}{\mathbf{U}_{\mathcal{N}}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{S}_{\rho\times \rho} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % V* \left[ \begin{array}{c} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{V}_{\mathcal{N}}}^{*} \end{array} \right] \\ % \end{align} $$ The Moore-Penrose pseudoinverse is $$ \begin{align} \mathbf{A}^{+} &= \mathbf{V} \, \Sigma^{+} \, \mathbf{U}^{*} \\ % &= % V \left[ \begin{array}{cc} \color{blue}{\mathbf{V}_{\mathcal{R}}} & \color{red}{\mathbf{V}_{\mathcal{N}}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{S}^{-1}_{\rho\times \rho} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % U* \left[ \begin{array}{c} \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{U}_{\mathcal{N}}}^{*} \end{array} \right] \\ % \end{align} $$
When $\rho<\min\left(m,n\right)$, both null spaces are nontrivial.
The product matrices are $$ \begin{align} % \mathbf{A}^{+} \mathbf{A} &= % V \left[ \begin{array}{cc} \color{blue}{\mathbf{V}_{\mathcal{R}}} & \color{red}{\mathbf{V}_{\mathcal{N}}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{I}_{\rho} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % V* \left[ \begin{array}{c} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{V}_{\mathcal{N}}}^{*} \end{array} \right] % = % \color{blue}{\mathbf{V}_{\mathcal{R}}} \color{blue}{\mathbf{V}^{*}_{\mathcal{R}}} % \ne % \mathbf{I}_{n} \\ %%%%% % \mathbf{A} \mathbf{A}^{+} &= % U \left[ \begin{array}{cc} \color{blue}{\mathbf{U}_{\mathcal{R}}} & \color{red}{\mathbf{U}_{\mathcal{N}}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{I}_{\rho} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % U* \left[ \begin{array}{c} \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{U}_{\mathcal{N}}}^{*} \end{array} \right] % = % \color{blue}{\mathbf{U}_{\mathcal{R}}} \color{blue}{\mathbf{U}^{*}_{\mathcal{R}}} % \ne % \mathbf{I}_{m} % \end{align} $$
_Row rank deficient, full column rank_
$$ \mathbf{A}\in\mathbb{C}^{m\times n}_{n}, \quad \mathbf{A} = % U \left[ \begin{array}{cc} \color{blue}{\mathbf{U}_{\mathcal{R}}} & \color{red}{\mathbf{U}_{\mathcal{N}}} \end{array} \right] % Sigma \left[ \begin{array}{c} \mathbf{S}_{n\times n} \\ \mathbf{0} \end{array} \right] % V* \left[ \begin{array}{c} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \end{array} \right], \quad %%% \mathbf{A}^{+} = % U \left[ \begin{array}{cc} \color{blue}{\mathbf{V}_{\mathcal{R}}} \end{array} \right] % Sigma \left[ \begin{array}{c} \mathbf{S}^{-1}_{n\times n} & \mathbf{0} \end{array} \right] % V* \left[ \begin{array}{c} \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} & \color{red}{\mathbf{U}_{\mathcal{N}}}^{*} \end{array} \right] $$ When $\rho=n<m$, only the right null space is trivial: $$ \color{red}{\mathcal{N}\left( \mathbf{A}\right)} = \left\{ \mathbf{0} \right\} $$
The product matrices are $$ \begin{align} % \mathbf{A}^{+} \mathbf{A} &= % V \left[ \begin{array}{cc} \color{blue}{\mathbf{V}_{\mathcal{R}}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{I}_{n} \end{array} \right] % V* \left[ \begin{array}{c} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \end{array} \right] % = % \mathbf{I}_{n} \\ %%%%% % \mathbf{A} \, \mathbf{A}^{+} &= % U \left[ \begin{array}{cc} \color{blue}{\mathbf{U}_{\mathcal{R}}} & \color{red} {\mathbf{U}_{\mathcal{N}}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{I}_{n} \\ \mathbf{0} \end{array} \right] % U* \left[ \begin{array}{c} \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} \\ \color{red} {\mathbf{U}_{\mathcal{N}}}^{*} \end{array} \right] % = % \color{blue}{\mathbf{U}_{\mathcal{R}}} \color{blue}{\mathbf{U}^{*}_{\mathcal{R}}} % \ne % \mathbf{I}_{m} % \end{align} $$ This pseudoinverse is a left inverse: $\mathbf{A}^{+} = \mathbf{A}^{-L}$.
_Column rank deficient, full row rank_
$$ \mathbf{A}\in\mathbb{C}^{m\times n}_{m}, \quad \begin{align} \mathbf{A} = % U \left[ \begin{array}{cc} \color{blue}{\mathbf{U}_{\mathcal{R}}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{S}_{m\times m} & \mathbf{0} \end{array} \right] % V* \left[ \begin{array}{c} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{V}_{\mathcal{N}}}^{*} \end{array} \right] , \quad \mathbf{A}^{+} = % V \left[ \begin{array}{cc} \color{blue}{\mathbf{V}_{\mathcal{R}}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{S}^{-1}_{m\times m} \\ \mathbf{0} \end{array} \right] % U* \left[ \begin{array}{c} \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{U}_{\mathcal{N}}}^{*} \end{array} \right] \\ % \end{align} $$
When $\rho=m<n$, only the left null space is trivial, $$ \color{red}{\mathcal{N}\left( \mathbf{A}^{*}\right)} = \left\{ \mathbf{0} \right\} $$
The product matrices are $$ \begin{align} % \mathbf{A}^{+} \mathbf{A} &= % V \left[ \begin{array}{cc} \color{blue}{\mathbf{V}_{\mathcal{R}}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{I}_{m} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % V* \left[ \begin{array}{c} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \end{array} \right] = \color{blue}{\mathbf{V}_{\mathcal{R}}} \color{blue}{\mathbf{V}^{*}_{\mathcal{R}}} % \ne % \mathbf{I}_{n} \\ %%%%% % \mathbf{A}\, \mathbf{A}^{+} &= % U \left[ \begin{array}{cc} \color{blue}{\mathbf{U}_{\mathcal{R}}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{I}_{m} \end{array} \right] % U* \left[ \begin{array}{c} \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} \end{array} \right] % = \color{blue}{\mathbf{U}_{\mathcal{R}}} \color{blue}{\mathbf{U}^{*}_{\mathcal{R}}} % = \mathbf{I}_{m} % \end{align} $$ Therefore, the pseudoinverse is a right inverse: $\mathbf{A}^{+} = \mathbf{A}^{-R}$.
_Full column rank deficient, full row rank_
$$ \mathbf{A}\in\mathbb{C}^{m\times m}_{m}, \quad \begin{align} \mathbf{A} = % U \left[ \begin{array}{cc} \color{blue}{\mathbf{U}_{\mathcal{R}}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{S}_{m\times m} \end{array} \right] % V* \left[ \begin{array}{c} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \end{array} \right] , \quad \mathbf{A}^{+} = % V \left[ \begin{array}{cc} \color{blue}{\mathbf{V}_{\mathcal{R}}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{S}^{-1}_{m\times m} \end{array} \right] % U* \left[ \begin{array}{c} \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} \end{array} \right] \\ % \end{align} $$
When $\rho=m$, both null spaces are trivial, $$ \begin{align} % \color{red}{\mathcal{N}\left( \mathbf{A}\right)} &= \left\{ \mathbf{0} \right\} \\ % \color{red}{\mathcal{N}\left( \mathbf{A}^{*}\right)} &= \left\{ \mathbf{0} \right\} \end{align} % $$
The product matrices are $$ \begin{align} % \mathbf{A}^{+} \mathbf{A} &= \mathbf{I}_{m} \\ %%%%% % \mathbf{A}\, \mathbf{A}^{+} &= \mathbf{I}_{m} % \end{align} $$ Therefore, the pseudoinverse is both a right inverse: $\mathbf{A}^{+} = \mathbf{A}^{-R}$ and a left inverse: $\mathbf{A}^{+} = \mathbf{A}^{-L}$.