Linear Least Squares with Monotonicity Constraint

Question

I'm interested in the multidimensional linear least squares problem: $$\min_{x}||Ax-b||^2$$ subject to a monotonicity constraint for $x$, meaning that the elements of $x$ are monotonically increasing: $x_0 \leq x_1$, $x_1 \leq x_2$, ... , $x_{n-1} \leq x_n$.

I basically have two questions regarding this problem:

1.) Is there maybe literature regarding this problem out there? I wasn't able to find anything online so far.

2.) If not, is it maybe possible to rewrite my problem in such a way that i could use already existing methods like Non Negative Least Squares (NNLS) or a Constrained Least Squares (CLS) method?

Regarding the NNLS, I had the idea to formulate my problem in terms of an $\tilde{x} := (x_0, x_1-x_0,\; ...\;,x_n - x_{n-1})$ as this would also achieve monotonicity if every term in non negative, but I can't seem to do it, maybe I'm missing something here?

Many thanks in advance!

Answer 1

Let $L$ be an $n\times n+1$ matrix such that $$ L = \begin{pmatrix} -1 & 1 & 0 & ... &0 \\ 0 & -1 & 1 & ... &0 \\ & & \\ 0 & 0 & ...& -1 &1 \\ \end{pmatrix}$$

Then you can formulate this as a constrained least squares problem$$\min_{x}||Ax-b||^2\ s.t. Lx \geq 0$$

Answer 2

Your idea to reformulate the problem so that the variables are $x_0$ and $y_i = x_i - x_{i-1}$ for $i =1, \ldots, n$ will work. Let $y$ be the vector whose components are $x_0, y_1, \ldots, y_n$ and define $$ M = \underbrace{\begin{bmatrix} 1 & 0 & \cdots & 0 \\ 1 & 1 & \cdots & 0 \\ \vdots & & & \vdots \\ 1 & 1 & \cdots & 1 \end{bmatrix}}_{(n+1)\times(n+1)}. $$

Notice that $M y = x$. Expressed in terms of $y$, your optimization problem is to minimize $\| AM y - b \|^2$ subject to the constraint that $y_i \geq 0$ for $i = 1, \ldots, n$. In this reformulated problem, the optimization variable is the vector $y$.