Regularization for quadratic programming

72 Views Asked by At

I want to minimize $x$ in this equation with quadratic programming.

$$Z = \Phi x_0 + \Gamma U $$

Let's introduce a vector $R$ and I want to select $U$ as minimal as possible so $J$ will be minimized $$J = \frac{1}{2}||Z - R||^2$$

Here $\Phi$, $x_0$, $\Gamma$ and $R$ are known.

So I extend this problem:

$$J = \frac{1}{2}||Z - R||^2 = \frac{1}{2}||\Gamma U + \Phi x_0 - R||^2 = \frac{1}{2}(\Gamma U + \Phi x_0 - R)^TI(\Gamma U + \Phi x_0 - R)$$ Where $I$ is the identity matrix.

We continue...

$$J = \frac{1}{2}(\Gamma U + \Phi x_0 - R)^TI(\Gamma U + \Phi x_0 - R) = \frac{1}{2}U^T\Gamma^TI\Gamma U - (\Gamma^TI(R-\Phi x_0))^TU + \frac{1}{2}(R-\Phi x_0)^TI(R-\Phi x_0)$$

And then we write it on the QP-form:

$$J = \frac{1}{2}x^TQx + c^Tx$$

where $Q = \Gamma^TI\Gamma$ and $c = -\Gamma^TI(R-\Phi x_0)$. Forget about the constant $\frac{1}{2}(R-\Phi x_0)^TI(R-\Phi x_0)$

The constraints are:

$$\Gamma U \leq R - \Phi x_0$$ $$ U \geq 0$$

Question:

Where should I put the regulization?