What is the Fenchel dual problem for a general convex problem with constraints?
Precisely assume slater condition holds in following convex problem
$$\begin{matrix} \min & f(x) \\ s.t& g(x) \leq 0 \\ &Ax =b \end{matrix}$$
What would be the dual problem.
From Wikipedia I only see that Fenchel duality only works for convex unconstrained problems or at most with affine constraint. How can I incorporated nonlinear constrains into the dual problem?
By definition, the Fenchel dual is $$\begin{align} &\mu^T b + \inf_x (f+\lambda g - \mu^T A)(x)\\ = \; &\mu^Tb-\sup_x (0^Tx-f-\lambda g + \mu^T A)(x) \\ = \; &\mu^Tb-(f+\lambda g - \mu^T A)^*(0)\end{align}$$ To compute the conjugate of this sum, use the rules $$(f+\lambda g - \mu^T A)^*(y) = \inf_{y_1,y_2 : y_1 + y_2 + y_3 = y}f^*(y_1)+(\lambda g)^*(y_2) + (-\mu^TA)^*(y_3),$$ $$(\lambda g)^*(y_2)=\lambda g^*(y_2/\lambda)\text{, and}$$ $$(-\mu^TA)^*(y_3)=0 \text{ if } -A^T\mu=y_3.$$ After you substitute these last two formulas into the second one, you obtain the dual with variables $y_i$, $\lambda$ and $\mu$.