Minimizer of $\mathbb{E}\left[\left( g(\Theta) - \delta(X) \right)^2 \mid X \right]$

40 Views Asked by At

I have come across this one in Bayes estimators (just providing the context, nothing more required from Bayes estimation): the minimizer function $\delta_\Lambda(X)$ of $\mathbb{E}\left[\left( g(\Theta) - \delta(X) \right)^2 \mid X \right]$ is $\delta_\Lambda(X) = \mathbb{E}\left[ g(\Theta) \mid X \right]$.

My hunch (most probably the whole solution) is that this is equal to the Variance of $g(\Theta)$ given $X$ (which is independent of $\delta(X)$) plus another squared term which we zero out to get the result, but I would like to see a complete solution with potential pitfalls, since I might be missing some sweet, detailed spots.

1

There are 1 best solutions below

1
On BEST ANSWER

By basic algebra and linearity, as well as the “pull-out-what-you-know” properties of conditional expectations: \begin{align*} \mathbb E\left[\left(g(\Theta)-\delta(X)\right)^2\middle|X\right]&=\mathbb E\left[g(\Theta)^2\middle|X\right]-2\mathbb E\left[g(\Theta)\delta(X)\middle|X\right]+\mathbb E\left[\delta(X)^2\middle|X\right]\\ &=\mathbb E\left[g(\Theta)^2\middle|X\right]\underbrace{\!\vphantom2-2\delta(X)\mathbb E\left[g(\Theta)\middle|X\right]+\delta(X)^2}. \end{align*} The braced terms constitute a quadratic function in $\delta(X)$, whose minimum occurs at $\delta(X)=\mathbb E[g(\Theta)|X]$ for each realization of $X$. Note that the first term $\mathbb E[g(\Theta)^2|X]$ is not influenced by the choice of $\delta(X)$.