Penalty method in constraint optimization: Optimal penalty strength and update

66 Views Asked by At

I'm interested in numerically solving the constraint optimization problem of the form $$ \begin{cases}\ \displaystyle\min_{x\in\mathbb R^d} \|x\|_2^2 \\ \ f(x) = c \end{cases} $$ where $x\in\mathbb R^d$, $f\in C^1(\mathbb R^d)$, $c\in\mathbb R$ and $\| \cdot\|_2$ denotes Euclidean norm.

I wanted to use the penalty method which basically means one obtains a sequence $(x_n)_n\subset\mathbb R^d$ where $x_n$ is the solution of $$ \begin{cases}\ \displaystyle\min_{x\in\mathbb R^d} g_n(x) \\ \ x_{\text{initial}}=x_{n-1} \end{cases} $$ with $$g_n(x) = \|x\|_2^2\ +\ \lambda\mu^n| f(x) -c|^2 $$ for some initial initial condition $x_{-1}$ chosen. If $(x_n)$ "converges", i.e. $|x_m - x_{m-1}| < \epsilon$ for some $m\in\mathbb N$ and some predefined stopping criterion $\epsilon$, I stop this process and return $x_{m}$ as my solution to the problem.

In principle this method is working quite well but my question is now the following: Is there a rule of thumb or reference how to choose the initial penalty strength $\lambda$ and the update parameter $\mu$ in terms of the parameters \begin{align} & \text{distance of initial condition to origin:} \ \ \|x_{-1}\|_2 \\ & \text{value of penalty term at intial condition:} \ \ f(x_{-1}) \\ & \text{dimension:} \ \ d \end{align} or any parameter I haven't thought of.