Matrix least-squares problem with lower triangular matrix constraint

1.3k Views Asked by At

Given the matrices $\mathbf{A} \in \mathbb{R}^{N \times N}$ and $\mathbf{B} \in \mathbb{R}^{M \times N}$,

\begin{align} \arg \min_{\mathbf{X} \in \mathbb{R}^{N \times M}} \| \mathbf{A} - \mathbf{X} \mathbf{B} \|_\mathrm{F} \end{align}

where $\|\cdot\|_\mathrm{F}$ is the Frobenius norm operator. The above problem can be rewritten as

\begin{align} \arg \min_{\mathrm{vec}(\mathbf{X})} \mathrm{vec}(\mathbf{X})^T (\mathbf{B}\mathbf{B}^T \otimes \mathbf{I}) \mathrm{vec}(\mathbf{X}) - 2 \mathrm{vec}(\mathbf{A} \mathbf{B}^T)^T \mathrm{vec}(\mathbf{X}). \end{align}

where $\otimes$ is the Kronecker product. The above optimization can be solved easily as it is a quadratic program with no constraints. Suppose, we are given prior information that $\mathbf{X}$ is a lower-triangular matrix, how do I impose it as an equality constraint in the form of $\mathbf{C} \mathrm{vec}(\mathbf{X}) = \mathrm{vec}(\mathbf{Y})$ where $\mathbf{C} \in \mathbb{R}^{MN \times MN}$ and $\mathrm{vec}(\mathbf{Y})$ is the vectorized lower-triangular entries of $\mathbf{X}$? In other words, how to determine the entries of matrix $\mathbf{C}$?

Note that I can use CVX in MATLAB to solve this but when the dimensions of the matrices are large, then CVX takes a lot of time for computing.

1

There are 1 best solutions below

6
On BEST ANSWER

The problem is given by:

$$ \arg \min_{X \in \mathcal{T} } \frac{1}{2} {\left\| X B - A \right\|}_{F}^{2} $$

Where $ \mathcal{T} $ is the set of Lower Triangular Matrices.

The set $ \mathcal{T} $ is a Convex Set.
Moreover, the orthogonal projection onto the set of a given matrix $ Y \in \mathbb{R}^{m \times n} $ is easy:

$$ X = \operatorname{Proj}_{\mathcal{T}} \left( Y \right) = \operatorname{tril} \left( Y \right) $$

Namely, zeroing all elements above the main diagonal of $ Y $.

By utilizing the Projected Gradient Descent it is easy to solve this problem:

$$ \begin{align*} {X}^{k + 1} & = {X}^{k} - \alpha \left( X B {B}^{T} - A {B}^{T} \right) \\ {X}^{k + 2} & = \operatorname{Proj}_{\mathcal{T}} \left( {X}^{k + 1} \right)\\ \end{align*} $$

enter image description here

The full MATLAB code with CVX validation is available in my StackExchnage Mathematics Q2876283 GitHub Repository.

The solution is very similar to the solution in Q2421545 - Solve Least Squares (Frobenius Norm) Problem with Diagonal Matrix Constraint.

Remark
I think you can also get a closed form solution for each element in $ X $ if you go through deriving the derivative with respect to each element $ X $.
Another approach would be developing the Linear Operator which operates on $ \frac{ \left( n - 1 \right) n }{2} $ elements and creates an $ n \times n $ Triangular Matrix.