Minimum matching convolution (part II)

79 Views Asked by At

We assume we are working in $\mathcal{H}(\mathbb{R}^n)$, the space of real symmetric matrices. We define the partial order $\ge$ defined as $\Sigma_1\ge \Sigma_2$ iff $\Sigma_1-\Sigma_2$ is in $\mathcal{H}^+(\mathbb{R}^n)$, the semi-definite positive matrices in $\mathcal{H}(\mathbb{R}^n)$. The problem I would like to solve is (assuming $x\ge 0$) $$ \min_{x + \Sigma_1-\Sigma_2 \ge 0} ||x+\Sigma_1||^2 ~~,\tag{1}$$ where the norm is the Frobenius norm, $\Sigma_1, \Sigma_2\ge 0$. This problem is related with finding the Gaussians that match two given Gaussian distributions that minimize the size of the final image.

Let us define $U(A,B)=\lbrace X:X\ge A, X\ge B\rbrace$. Problem (1) is equivalent to $$ \min_{z\in U(\Sigma_1,\Sigma_2)} ||z||^2 ~~.\tag{1a}$$ In this post I asked about a minimization problem similar to (1a). That problem can be written in many equivalent forms, one of this is $$\min_{z\in U(\Sigma_1,\Sigma_2)}||z-\Sigma_1||^2~~.\tag{2}$$ The solution to problem (2) is $z=\Sigma_2+(\Sigma_1-\Sigma_2)_+$, where $(\cdot)_+$ denotes the positive semidefinite part. We also define $S_-=(-S)_+$ and $|S|=S_++S_-$. Note that $S=S_+-S_-$ and that $|S|=|-S|$. Since in general $$B+(A-B)_+=\frac{A+B+|A-B|}{2}~~,$$ we have that the solution of problem (2) is also the solution of $$\min_{z\in U(\Sigma_1,\Sigma_2)}||z-\Sigma_2||^2~~.\tag{2a}$$

Is there an explicit solution to problem (1)?

1

There are 1 best solutions below

0
On

We have at least an implicit solution to problem (1). In the $n=2$ case (which is the one I am mostly interested), possible solutions are parametrized by a single parameter. Therefore, the final solution can be obtained easily through numerical minimization.

This problem illustrates fundamental differences in the geometry of the space of matrices endowed with the Loewner partial order and the Frobenius metric and $\mathbb{R}^n$ endowed with the partial order induced by the positive orthant (that is, $p\ge q$ ssi $p_i\ge q_i, i=1,\ldots,n$) and the Euclidean metric. The following picture shows what problems (1) and (2) would be like in $\mathbb{R}^2$. $U(\Sigma_1,\Sigma_2)$ and the minimization problems in $R^2$

In this picture, problems (2) and (2a) correspond to minimizing over the orange region ($U(\Sigma_1,\Sigma_2)$) the norms of the vectors that start on $\Sigma_1$ and $\Sigma_2$, respectively. The minimization of the norm of the vector that start in the origin corresponds to problem (1a). As shown in the second part of the OP, problems (2) and (2a) have the same solution. The solution to Problem (1a) is not this same solution in general. The crucial issue is that the $U(\Sigma_1,\Sigma_2)$ is not a cone because the matrices space is not a lattice.

When the solutions coincide? When one of the matrices is greater or equal than the other, say, $\Sigma_1\ge\Sigma_2$ then the solution of problems (1a), (2) and (2a) is $\Sigma_1$ and the solution to problem (1) is $x=0$.

The geometry of $U(\Sigma_1,\Sigma_2)$ was studied by Ando (1993, Acta Sci. Math. (Szeged) 57, 3–10). Specifically, he studied the minimal points of $U(\Sigma_1,\Sigma_2)$. Because if $x\ge y$ then $||x||\ge||y||$, we expect that a solution of problem (1a) to be a minimal point of $U(\Sigma_1,\Sigma_2)$. Ando (1993) gives a explicit representation of these minimal points. I do not know if such an explicit solution for Problem (1) exists in the general case.

For the $n=2$ case, the representation of all minimal points is given in the following way. Let $|\Sigma_1-\Sigma_2|=U^T\Lambda U$ and the matrix $$\mathcal{D}(\kappa):=\frac{1}{1-\kappa^2}\begin{pmatrix}1+\kappa^2 & 2\kappa \\ 2\kappa & 1+\kappa^2\end{pmatrix}~~,$$ defined for each $|\kappa|<1$. Then the minimal points of $U(\Sigma_1,\Sigma_2)$ in the $n=2$ case are given by $$ Z_m(\kappa)=\frac{1}{2}\left(\Sigma_1+\Sigma_2+U^T\Lambda^{1/2}\mathcal{D}(\kappa)\Lambda^{1/2}U\right)~~.\tag{R1}$$ We have that $Z_m(0)=(A+B+|A-B|)/2$, that is, the solution to Problem (2).

Example: $$\Sigma_1=\begin{pmatrix}7&-1\\-1&2\end{pmatrix}~~,\Sigma_2=\begin{pmatrix}3&2\\2&3\end{pmatrix} $$ The following picture shows the norms of $||Z_m(\kappa)||$ and of $||Z_m(0)||$. Note that the optimum occurs for $\kappa\approx0.018$. That is, the narrowest gaussian you can match via convolution is not necessarily the one resulting from convolving with the narrowest gaussian that attains the match.

enter image description here