Rank of a particular type of block matrix

1.2k Views Asked by At

Suppose that $A_1$ and $A_2$ are both of size $(n+1)\times m$, with $m\ge 2$, and that $B_1$ and $B_2$ are both of size $(n+1)\times n$. In addition, it is known that both $B_1$ and $B_2$ are full rank, hence rank($B_1$)$=$rank($B_2$)$=n$.

Consider the following block matrix $$M=\begin{pmatrix} A_1 & B_1 & 0\\ A_2 & 0 & B_2 \end{pmatrix}$$

The question is: What can be said about the rank of $M$? More precisely, under what conditions does it hold that $M$ has the largest possible rank (i.e., rank($M$)$=2n+2$)?

This question is motivated by trying to determine the conditions under which extra equations with new variables introduce new information that can be helpful to solve a system of equations. For example, suppose that $z_1$, $z_2$, $x$ and $y$ are unknowns and consider the equations $$z_1+z_2+x=4$$ $$z_1+2z_2+3x=2.$$ Here we have an undetermined system with infinite solutions. However, if we also add the equations $$z_1+z_2+5y=1$$ $$z_1+2z_2+8y=4$$ then we will have a system of 4$\times$4 of full rank with a unique solution. On the other hand, suppose that instead of the above equations we add $$z_1+z_2+y=4$$ $$z_1+2z_2+3y=2.$$ Here we will have a system of 4$\times$4 of rank 3, and hence an infinite number of solutions. It can be concluded that the first set of equations has more "information" than the second one.

What I have tried: There are some theorems on the rank of block matrices, such as the following one: $$rank([A,B])=rank(B)+rank(A-BB^+A),$$ where $B^+$ is the pseudoinverse of $B$. However, this type of results are not that helpful because one then has to compute the ranks of sum of matrices and pseudoinverses which in general is non-trivial (as far as I know). For example, using the above theorem, and setting $$ A=\begin{pmatrix} A_1\\ A_2\end{pmatrix}, B=\begin{pmatrix} B_1 & 0\\ 0 & B_2\end{pmatrix} $$ one has that $$rank(M)=2n+rank(C)$$ where $C$ is $$C=\begin{pmatrix} A_1-B_1(B_1^\top B_1)^{-1}B_1^\top A_1\\ A_2-B_2(B_2^\top B_2)^{-1}B_2^\top A_2 \end{pmatrix},$$ and computing the above rank does not seem straightforward.

1

There are 1 best solutions below

1
On BEST ANSWER

We can quickly observe that $$ \operatorname{rank}\pmatrix{B_1&0\\0&B_2} = \operatorname{rank}(B_1) + \operatorname{rank}(B_2) = 2n $$ It follows that the rank of $M$ is at least $2n$, though this also follows from your theorem.


Let's take a closer look at that matrix $C$. The question we have is: when will its rank be at least $2$? Note that for any vector $x$, $(I - B(B^TB)^{-1}B^T)x$ is the error in the least squares solution to $By = x$. That is, if $\hat z$ is the least squares solution to $B\hat z = x$, then $$ (I - B(B^TB)^{-1}B^T)x = x - B\hat z $$ In particular, note that this product will be non-zero if the equation has an exact solution, and $(I - B(B^TB)^{-1}B^T)x$ will necessarily lie in the orthogonal complement to the column space of $B$, which is to say that it will be in the nullspace of $B^T$. Since $B$ has rank $n$, this null-space has dimension $1$. So, $(I - B(B^TB)^{-1}B^T)x$ will always be equal to a multiple of $v$ for some fixed non-zero vector $v$ (the basis of the nullspace of $B^T$).

What does this have to do with $C$? If we label the columns of $A_1$ as $x_1,x_2,\dots,x_n$, then we have $$ A_1 - B_1(B_1^TB_1)^{-1}B_1^TA_1 = (I - B_1(B_1^TB_1)^{-1}B_1^T)A_1 = \\ \pmatrix{(I - B_1(B_1^TB_1)^{-1}B_1^T) x_1 & \cdots & (I - B_1(B_1^TB_1)^{-1}B_1^T)x_m} = \\ \pmatrix{\alpha_1 v & \cdots & \alpha_m v} = \\ v \pmatrix{\alpha_1 & \cdots & \alpha_m} $$ For some scalars $\alpha_i$, where $\alpha_i = 0$ if and only if $B_1z = x_i$ has a solution. Similarly, $$ A_2 - B_2(B_2^TB_2)^{-1}B_2^TA_2 =\\ w \pmatrix{\beta_1 & \cdots & \beta_m} $$ In fact, we can write the full matrix $C$ as $$ C = \pmatrix{v&0\\0&w}\pmatrix{\alpha_1 & \cdots & \alpha_m\\ \beta_1 & \cdots & \beta_m} $$ Since the left matrix has full column-rank, we have $$ \operatorname{rank}(C) = \operatorname{rank}\pmatrix{\alpha_1 & \cdots & \alpha_m\\ \beta_1 & \cdots & \beta_m} $$


This is still pretty difficult to analyze, but here are some quick consequences:

  • If $M$ has full rank, there must be at least one pair $x_i,y_j$ with $i \neq j$ such that $B_1z = x_i$ and $B_2z = y_i$ have no solution.
  • $M$ will have full rank if there is a pair $i \neq j$ such that
    • $B_1z = x_i$ has no solution
    • $B_1z = y_i$ has a solution
    • $B_1z = x_j$ has a solution
    • $B_1z = y_j$ has no solution