the Bayes rule with a general form (Young textbook problem 3.3)

88 Views Asked by At

Hi so I am trying to find the bayes rule with a loss function defined as for a general estimator. we just need to find the general form of the Bayes rule for this loss function. let the prior be defined as $\pi(\theta)$ and posterior as $\pi(\theta|x)$. Let the bayes risk be denoted as $r(\pi,d)$ and we need to take the minimum of r

$$ L(\theta,d) = \left\{ \begin{array}{ll} a(\theta-d) & d\leq \theta \\ b(d-\theta) & b>\theta \\ \end{array} \right. $$

so using the defintion of risk and the Bayes rule which integrates over $\theta$

\begin{equation} R(\theta,d) = \int_x L(\theta,d(x))f(x| \theta)dx \\ \implies r(\pi,d)_{bayes} = \int_\theta \int_x L(\theta,d(x))f(x|\theta) \pi(\theta) dx d\theta \\ = \int_\theta L(\theta,d(x))\pi(\theta|x) d\theta \space \space \space(1) \end{equation}

So we need minimize equation (1) by taking the derivative with respect to d to minimize the decision function (by definition of Bayes).

Plugging in the loss function into equation 1 and taking the derivative wrt d (our decision function)

\begin{equation} \frac{\partial}{\partial d} (\int_{-\infty}^d b(d-\theta)\pi(\theta|x) d\theta + \int_{d}^\infty a(\theta-d)\pi(\theta|x) d\theta ) ) \end{equation}

My question is to double check my use of the Leibnitz rule to take the derivative of the integrals with the function in the limits

using the Leibnitz rule I have

\begin{equation} = b(d-d)\pi(d|x) + \int_{-\infty}^d b*\pi(\theta|x)d\theta + a(d-d)\pi(d|x) + \int_d^\infty -a*\pi(\theta|x) d\theta \\ \end{equation}

Some terms drop out, and we can simplify this equation a bit, but is this the correct approach using Leibnitz rule? thank you!

1

There are 1 best solutions below

0
On

Your argument for (1) is wrong; you're right that we want to minimise the posterior risk

$$\rho(x, \pi, d) = \int L(\theta,d(x))\pi(\theta|x) d\theta,$$

but that's a different quantity from the Bayes risk $$r(\pi, d) = \int \rho(x, \pi, d) p_\pi(x) dx.$$ If we minimise the posterior risk $\rho$ for each $x$, we'll also minimise the Bayes risk $r$.

That aside, yes, you have the right approach. Your final equation is correct as stated, but you can simplify the argument using the measure-theoretic version of the Leibnitz rule,

$$\frac{\partial}{\partial d}\int_{-\infty}^\infty L(\theta, d) \pi(\theta \mid x)d\theta = \int_{-\infty}^\infty\frac{\partial L(\theta, d)}{\partial d}\pi(\theta \mid x)d\theta,$$ since the loss $L$ is differentiable in $d$ almost everywhere, and has locally integrable derivative.