Let $f: \mathbb{R}^n \rightarrow [0,\infty)$ be a convex function (not necessarily strictly convex) and let $g: \mathbb{R}^n \rightarrow [0,\infty)$ be a strictly convex function. Suppose that the set of minimizers of $f$, defined as $M = \text{arg}\min_{x \in \mathbb{R}^n} f(x)$ is nonempty and bounded. Consider a sequence of positive regularization parameters $\{ \delta_k \}_{k=1}^\infty$. Define the sequence of solutions $\{ x_k \}_{k=1}^\infty$ by $$ x_k = \text{arg} \min_{x \in \mathbb{R}^n} f(x) + \delta_k g(x) \enspace. $$ We also define $x^*$ to be the point minimizing $g$ amongst all minimizers of $f$. That is, $$ \begin{aligned} x^* = \text{arg}\min_{x \in \mathbb{R}^n} \quad & g(x)\\ \textrm{s.t.} \quad & f(x) = \inf \{ f(y) : y \in \mathbb{R}^n \} \\ \end{aligned} $$ Note that by strict convexity of $g$, each point in the sequence $\{ x_k \}$ as well as the point $x^*$ are well-defined.
Question: Under what conditions does the sequence $x_k$ converge? Under what conditions does the sequence converge to $x^*$?
So far, I am able to show (without appealing to any properties of $f$ and $g$) that
- $\lim_{k \rightarrow \infty} f(x_k) = f(x^*)$
- $\lim_{k \rightarrow \infty} g(x_k) = g(x^*)$
However, this is convergence of the function values and does not show that $x_k \rightarrow x^*$. Note that because $\mathbb{R}^n$ is not bounded, we can't directly appeal to boundedness in order to show that there exists a subsequence that converges, but it still should be possible to establish boundedness of the sequence $x_k$ perhaps using properties of $f$ and $g$.
I think that some statement like this must be known in the study of barrier functions for optimization algorithms. Part of my motivation is to formalize some of the arguments in Proposition 4.1 of Computational Optimal Transport by Peyré and Cuturi.
EDIT: The proof of Peyré and Cuturi is totally correct, I just didn't realize the following crucial fact: Let $\{x_k\}$ be a sequence in a compact metric space. If all convergent subsequences have the same limit point $x^*$, then the entire sequence converges to $x^*$. I will write an answer in a few days.