On $\lim\limits_{v \to \infty} \underset{\theta \in \Theta}{\operatorname{argmax}} \int_{\| \theta - z \| < \frac{1}{v}} \pi(z|x) \, {\rm d} z$

Question

I am studying a paper¹ and do not understand the first equality in equation (6) page 4 (page 3 on document).

---

---

I already posted once this question under the tag of real analysis since I thought it must be some result related to that, but no one could answer me. I therefore assume it must have something to do with a property of the density function or Bayes estimator or something.

---

1. Robert Bassett, Julio Deride, Maximum a posteriori estimators as a limit of Bayes estimators, 2018.

Answer 1

Let me impose a mild assumption that $\pi(\theta|x)$ lies in the space $C_0(\mathbb{R}^n)$ of continuous functions $f(\theta)$ on $\mathbb{R}^n$ that vanishes as $\|\theta\|\to\infty$ for each $x \in \mathcal{X}$. Then by fixing $x$ and writing $ f(\theta) = \pi(\theta|x) $ to suppress $x$ from notation, the question boils down to:

Question. Suppose $f \in C_0(\mathbb{R}^n)$ is a PDF. Is it true that $$ \lim_{\varepsilon\to0^+} \mathop{\rm argmax}_{\theta\in\mathbb{R}^n} \int_{\|z-\theta\|<\varepsilon} f(z) \, \mathrm{d}z = \mathop{\rm argmax}_{\theta\in\mathbb{R}^n} f(\theta) \quad ? \tag{1} $$

(I replaced $1/\nu$ by $\varepsilon$.) Since $\mathrm{argmax}$ is not affected by scaling of functions, we may replace the integral in the LHS of $\text{(1)}$ by the average value $\bar{f_{\varepsilon}}$ of $f$ over the ball $\mathsf{B}(\theta,\varepsilon)$ of radius $\varepsilon$ about $\theta$:

$$ \bar{f_{\varepsilon}}(\theta) := \frac{1}{\operatorname{Vol}(\mathsf{B}(\theta,\varepsilon))} \int_{\mathsf{B}(\theta,\varepsilon)} f(z) \, \mathrm{d}z $$

Then the reformulation of $\text{(1)}$,

$$ \lim_{\varepsilon\to0^+} \mathop{\rm argmax}_{\theta\in\mathbb{R}^n} \bar{f_{\varepsilon}}(\theta) \stackrel{?}= \mathop{\rm argmax}_{\theta\in\mathbb{R}^n} f(\theta), \tag{2} $$

makes much more sense, as the average value $\bar{f_{\varepsilon}}(\theta)$ should (and in fact, does) converge to $f(\theta)$ as the radius $\varepsilon$ shrinks to $0$, i.e., $\varepsilon \to 0^+$. Indeed, we can prove the following:

Claim. Under the same assumption as in the question, the equality $\text{(2)}$ holds in Hausdorff convergence of compact sets.

Since it is never guaranteed that $\mathrm{argmax}$ is a singleton (and hence can be regarded as a single value), I invoked the notion of convergence of sets in this statement. This is equivalent to the usual convergence if all the $\mathrm{argmax}$'s are guaranteed to be singletons.

Let me skip the proof for this moment, but I will provide one at a later time if you are interested.

Answer 2

I understand it to be saying the following: We are integrating over the region where $||\theta - z|| < \frac{1}{\nu}$. As $\nu \to \infty$, $\frac{1}{\nu} \to 0$, so the region where $||\theta - z|| < \frac{1}{\nu}$ approaches a single value of $z$, namely $z = \theta$.