Minimization involving equality constraints

388 Views Asked by At

I am trying to find closed form solution to following problem

\begin{equation} \begin{array}{c} \underset{\mathbf{x},\mathbf{y}}{\text{minimize}} \hspace{4mm} \big(\left( \mathbf{y}^T V^{(1)}\mathbf{x} \right)^2 + \left( \mathbf{y}^T V^{(2)}\mathbf{x} \right)^2\big) \\ s.t \hspace{10mm}\mathbf{x}^T\mathbf{e_x} = 1 \\ \hspace{17mm}\mathbf{y}^T\mathbf{e_y} = 1, \end{array} \end{equation}

where $\mathbf{x},\mathbf{e_x} \in \mathbb{R}^{m}$, $\mathbf{y},\mathbf{e_y} \in \mathbb{R}^{n}$, and $\mathbf{V}^{(i)} \in \mathbb{R}^{n\times m},\forall i$ . And

\begin{align} \mathbf{e_x} = \left[ \begin{array}{ccc} 1 \space\space 0 \space\space 0 \space\space \cdots 0 \end{array} \right]^T\\ \mathbf{e_y} = \left[ \begin{array}{ccc} 1 \space\space 0 \space\space 0 \space\space \cdots 0 \end{array} \right]^T \end{align}

My approach: Rewrite unconstraint objective Lagrange multipliers that take care of equality constraints

\begin{equation} \begin{array}{c} \underset{\mathbf{x},\mathbf{y}}{\text{minimize}} \hspace{4mm} \big(\left( \mathbf{y}^T V^{(1)}\mathbf{x} \right)^2 + \left( \mathbf{y}^T V^{(2)}\mathbf{x} \right)^2\big) + \color{red}{\mathcal{v}_1}(\mathbf{x}^T\mathbf{e_x}-1) + \color{red}{\mathcal{v}_2}(\mathbf{y}^T\mathbf{e_y}-1) \end{array} \end{equation}

Question

  1. IS this reformulation correct?
  2. If yes, then how should I proceed with solving it.
1

There are 1 best solutions below

1
On

$\color{red}{Partial Answer}$ Eliminating equality constraints

Define following matrices for equality constraints

\begin{align} \mathbf{A}_{2\times(n+m)} &= \left[ \begin{array}{ccc} \mathbf{e_x} \space\space \mathbf{0}_{1\times n}\\ \mathbf{0}_{1\times m} \space\space \mathbf{e_y} \end{array} \right], \space \mathbf{z}_{(n+m)\times 1} = \left[ \begin{array}{ccc} \mathbf{x}\\ \mathbf{y} \end{array} \right],\space \mathbf{b}_{2\times 1} = \left[ \begin{array}{ccc} 1\\ 1 \end{array} \right]\\ \mathbf{A}\mathbf{z}&=\mathbf{b} \end{align}

Also, we need two more matrices $\mathbf{P},\mathbf{R}$ for the objective function such that $\mathbf{z}^T\mathbf{P}=\mathbf{y}^T$ and $\mathbf{R}\mathbf{z}=\mathbf{x}$

\begin{align} \mathbf{P}_{(n+m)\times(n)} &= \left[ \begin{array}{ccc} \mathbf{0}_{(m\times n)}\\ \mathbf{I}_{(n\times n)} \end{array} \right]\\ \mathbf{R}_{(m)\times(n+m)} &= \left[ \begin{array}{ccc} \mathbf{I}_{(m\times m)} \space \mathbf{0}_{(m\times n)} \end{array} \right] \end{align}

With these definition, rewrite objective function as

\begin{equation} \begin{array}{c} \underset{\mathbf{z}}{\text{minimize}} \hspace{4mm} \big(\left( \mathbf{z}^T \mathbf{P}V^{(1)}\mathbf{R}\mathbf{z} \right)^2 + \left( \mathbf{z}^T \mathbf{P}V^{(2)}\mathbf{R}\mathbf{z} \right)^2\big) \\ s.t \hspace{10mm}\mathbf{A}\mathbf{z}=\mathbf{b} \end{array} \end{equation}

Which is in the same form as

\begin{equation} \begin{array}{c} \underset{\mathbf{z}}{\text{minimize}} \hspace{4mm} f(\mathbf{z}) \\ s.t \hspace{10mm}\mathbf{A}\mathbf{z}=\mathbf{b} \end{array} \end{equation}

and follow this method on slide 11-4 http://see.stanford.edu/materials/lsocoee364a/11EqualityMin.pdf enter image description here