I am self studying Bayes Decision theory from these lecture notes page 30 / 31 and there is a step a struggle to understand mathematically
Background context
Given Bayes risk defined as:
$$ r_B(\pi, \hat \theta) = \int_{\Theta} R(\theta, \hat \theta) \ \pi(\theta) \ d \theta$$
- Prior distribution $\theta \sim \pi(\theta)$
- Distribution of data given $\theta$ $z | \theta \sim f(z | \theta)$
- Marginal distribution of $z$: $m(z) = \int f(z | \theta) \pi (\theta) \ d\theta$
- Posterior distribution of $\theta$ $\pi(\theta | z) = \frac{f(z | \theta) \pi(\theta)}{m(z)}$
- Frequentist risk $R$: $R(\theta, \hat \theta) = E[L(\theta, \hat \theta)| \theta]$
We can express Bayes Risk in terms of the posterior risk, a function of $z$
$$ r_B(\pi, \hat \theta) = \int_{\mathcal{Z}} r(\hat \theta | z) m(z) dz$$
Which means that the Bayes decision rule can be obtained as
\begin{aligned} r_b(\pi, \hat \theta_\pi^{Bayes}) &= \inf_{\hat \theta} r_b(\pi, \hat \theta) \\ &= \inf_{\hat \theta} \int_{\mathcal{Z}} r(\hat \theta | z) m(z) dz \end{aligned}
Step I don't grasp
The above theorem implies that the Bayes rule can be obtained by taking the Bayes action for each particular $z$. For each fixed $z$ we choose $\hat \theta(z)$ to minimize the posterior risk $r(\hat \theta | z)$.
$$ \arg \min_{\hat \theta} \ r(\hat \theta | z) = \int L(\theta, \hat \theta(z)) \pi(\theta | z) \ d\theta $$
This guarantees us to minimize the integrand at every $z$ and hence minimize Bayes risk.
How do we get from the inf to the argmin and why does one minimize the other ? I feel this is a stupid question but I am missing something... thanks !