I want to solve the following constraint optimization problem for many different $\alpha$. $$\theta^* = \underset{\theta}{\text{argmin}}\, f(\theta) \text{ s.t. } g(\theta) \leq \alpha$$ Motivated by the duality principle, I try to solve this instead for various $\mu$: $$\hat\theta = \underset{\theta}{\text{argmin}}\, \left( f(\theta) + \mu g(\theta) \right)$$ I am trying to get a better handle on when this approach works or fails.
In particular, can we say that it works when $(f(\theta^*), g(\theta^*))$ is convex, i.e., will the curve $(f(\hat\theta), g(\hat\theta))$ be equivalent? Can we prove that it fails when $(f(\theta^*), g(\theta^*))$ is concave?