I was trying to solve the optimization problem:
$$ \min \| p \|^2 \text{ subject to } l(p) = \epsilon$$
or to simplifying notation:
$$ \min f(x) \text{ subject to } g(x) = 0 $$
using Lagrange multipliers. Then I noticed that using the auxiliary equation we get:
$$ \min f(x) + \lambda g(x)$$
of course that satisfies the usual constraint that the objective and constraint are parallel:
$$ \nabla f(x) = - \nabla \lambda g(x) $$
but so does:
$$ \min \alpha f(x) + g(x)$$
where we get:
$$ \alpha = \frac{1}{\lambda} $$
however this second seems like a very different thing because it corresponds to constraint optimization problem:
$$ \min g(x) \text{ subject to } f(x) = 0 $$
which seems to me a completely different problem but in the light of the auxiliary equation and the parallel line/gradient condition they seem equivalent, even symmetric. Is this symmetry expected or is it just special because I only have 1 objective and 1 constraint but if I included more constraints (or even inequalities) it wouldn't hold.
I tried it out and even though I don't really understand why this works with more than 1 constraint it seems the auxiliary equation would have more constraints with each a different Lagrange multiplier so we can't just do the $\alpha = \frac{1}{\lambda}$ trick:
$$ f(x) + \lambda_1 g(x) + \lambda_2 h(x) $$
so it seems we can't do that trick. Is this right or did I miss something simple? It seemed too simple so I just wanted to double check I was right.
This hasn't gotten so much to do with duality as it has with multiobjective optimization. You are treating $f$ and $g$ as objectives, and you can solve a multiobjective optimization problem with a weighted sum method or by constraining one of the objectives and optimizing the other(s).