KKT system with rank-deficient constraints

259 Views Asked by At

I have an optimization problem of the following form: $$ \begin{aligned} \operatorname*{minimize}_x & \quad \frac{1}{2}||x - a||^2 \\ \operatorname{subject~to} & \quad D^Tx = b \end{aligned} $$ I attempted to solve it, but I get results which do not satisfy the constraints. I am going to show how I solved it, and would like to ask your help in finding what is wrong.

The resulting KKT system is: $$ \begin{aligned} &x + D \lambda = a \\ &D^T x = b \end{aligned} $$

In my case, the matrix $D$ is rank-deficient, huge, and I do not directly represent it. Instead, I have functions which can efficiently compute $Dz$ and $D^T z$. In addition, I can quickly compute $(D^T D)^\dagger z$ ($\dagger$ is the pseudo-inverse) using a very fast algorithm.

Using the first equation I get: $$ x = a - D \lambda $$ I substitute this $x$ into the second equation: $$ \begin{aligned} (D^T D) \lambda = D^T a - b \end{aligned} $$ Now I solve the system above by using my algorithm to compute $\lambda = (D^T D)^\dagger (D^T a - b)$. After computing $\lambda$ I can compute $x = a - D \lambda$.

As stated at the beginning, I am getting $x$ which does not satisfy the constraints, and I would like to know why. What am I missing?


Update

As pointed out in the comments, there might be no solution to the system of constraints due to rank-deficiency. However, I found a way to construct vectors $b$ such that it will not happen. Thank you for the comments.