Prove that the dual of the norm approximation problem has the given form.

84 Views Asked by At

Consider the norm approximation: $$ (P) \begin{cases} \min_{x \in \mathbb{R}^n} \Vert Ax - b \Vert \end{cases} $$

where $A \in \mathcal{M}_{m \times n}(\mathbb{R})$ and $b \in \mathbb{R}^m$.


Part (1) — Write $(P)$ with an equality constraint.

Attempt:

Since $\Vert Ax -b\Vert \geq 0$ then add the following constraint: $$ (P) \begin{cases} \min_{x \in \mathbb{R}^n} \Vert Ax - b \Vert,\\ \text{s.t.} \;\; Ax-b=0 \end{cases} $$ I am not sure if this is correct, or why this could be correct. It was pure intuition because the smallest value the norm can attain is $0$.


Part (2) — Prove that the dual of $(P)$ is the following: $$ (P^*) \begin{cases} \max_{\nu} \; \langle b, \nu \rangle, \\ \text{s.t. } \; A^\top \nu = 0, \;\; \Vert \nu \Vert_* \leq 1. \end{cases} $$ The dual norm is defined as $\Vert \nu \Vert_* = \sup_{\Vert x \Vert \leq 1} \langle \nu, x\rangle $.

Attempt:

The Lagrangian is $L(x,\nu) = \Vert Ax -b\Vert + \langle \nu,Ax-b \rangle$

I know that the Lagrangian dual problem is defined as: $$ (P^*) \begin{cases} \max_{(\lambda,\nu)} \; \inf_{x \in \mathbb{R^n}} L(x,\lambda, \nu)\\ \text{s.t. } \; \lambda \geq 0 \end{cases} $$ Where $\lambda$ are the multipliers of the inequality constraints.

But I do not have any inequality constraints.
Can the $Ax - b = 0$ constraint be counted as $Ax - b \geq 0$ and $Ax - b \leq 0$?


Any help is greatly appreciated. I am really unsure of how to proceed.