Maximizing the distance between two matrices (Frobenius norm)

79 Views Asked by At

I need to maximize the distance between two matrices using the square of the Frobenius norm with a specific constraint. This is the subsequent problem:

$$\begin{array}{ll} \text{maximize} & \lVert M-A\rVert^2\\ \end{array}$$

$$\begin{array}{ll}\text{subject to}& M*X = I \\ \end{array}$$

Where $A$ and $M$ are matrices $m\times n$ and $X$ $n\times m$, where $m>n$ and $X$ and $A$ are known matrices.

How can I approach the problem? Either an analytical or a computational solution is valid for me.

Thanks in advance

A clarification: what I want to find is an $M$ that maximizes the objective function. A rectangular matrix has infinitely many inverses; the idea is to find the inverse that maximizes the objective function, hence the constraint.

1

There are 1 best solutions below

3
On

I assume that $X$ has rank $m$, so that there exists at least one matrix $M$ such that $MX = I$. I also assume that, since this objective function is unbounded over the feasible set, you're actually looking for the minimum instead.

I will use the following notation for the problem: $$ \min\|M - A\|^{2} \quad \text{s.t.} \quad MX = I. $$ The set of feasible matrices $M$ is an affine subspace of $\Bbb R^{m \times n}$. As such, this problem can be cast as a simple linear least squares problem.

One formula for the solution to this problem is given by $$ M_* = A[I - XX^+] + X^+, $$ where $X^+ = (X^TX)^{-1}X$ is the Moore-Penrose pseudoinverse of $X$.


Note that the equation $MX = I$ defines a linear system over the entries of $M$. The corresponding homogeneous system can be written as $MX = 0$. As such, the set of solutions to this equation can be written in the form $$ \{M_p + M_c : M_c X = 0\}, $$ where $M_p$ is any single matrix solving the equation $M_pX = I$ (i.e. $M_p$ is a "particular solution" to the original equation). One reasonable candidate for $M_p$ is the Moore Penrose pseudoinverse $M_p = X^+ = (X^TX)^{-1}X^T$.

As for the solutions to the equation $M_c X = 0$ (i.e. the "complementary" solutions to our original equation), we can characterize them as follows. Let $K$ denote be such that the columns of $K^T$ form a basis of the null space of $X^T$. Equivalently, $K$ is a matrix the largest possible number of rows while still being of full row-rank and satisfying $KX = 0$. One such matrix can be produced using the SVD of $X$.

If $r = \text{rank}(X)$, then $K$ has shape $(n-r) \times n$. Every complementary solution $M_c$ can be written in the form $JK$ for a suitable matrix $J$ of shape $m \times (n-r)$, and for every such matrix $J$ the matrix $JK$ is a complementary solution.

With all that established, the minimization can be recast as the following unconstrained optimization: $$ \min_{J \in \Bbb R^{m \times (n - r)}}\|(M_p + JK) - A\|^2 =\\ \min_{J \in \Bbb R^{m \times (n - r)}}\|JK - (A - M_p)\|^2. $$ By regarding this problem as independent minimizations over each row of $J$ and applying linear least squares, we can come to the conclusion that the minimizing solution $J$ is given by $$ J_* = (A - M_p)K^+ = (A - M_p)K^T(KK^T)^{-1}. $$ So, the minimizing solution $M$ to the original problem can be written as $$ M_* = M_p + J_*K = M_p + (A - M_p)K^T(KK^T)^{-1}K\\ = A[K^T(KK^T)^{-1}K] + M_p[I - K^T(KK^T)^{-1}K]. $$ As it turns out, the matrix $K^T(KK^T)^{-1}K$ can be directly computed from $X$: because $K^T(KK^T)^{-1}K$ is the orthogonal projection onto the column space of $K^T$, which is the nullspace of $X^T$, which is the orthogonal complement of the column space of $X$, we have $$ K^T(KK^T)^{-1}K = I - X(X^TX)^{-1}X^{T}. $$ With that, denoting $P_X = X(X^TX)^{-1}X^{T}$, we can rewrite our solution as $$ M_* = A[I - P_X] + M_p P_X. $$ This yields the correct answer regardless of which "particular solution" $M_p$ is used. However, if we take $M_p = X^+ = (X^TX)^{-1}X$ as I suggest above and note that $P_X = XX^+$, then the above formula can be rewritten as follows: $$ M_* = A[I - XX^+] + X^+X X^+ = A[I - XX^+] + X^+. $$