Mean Square Error of Monte Carlo

Question

Trying to develop the expression for the Mean Square Error (MSE) of Monte Carlo, I found myself a bit lost when going through a simple proof in the literature. I am working in the context of mathematical finance, where the aim is to find an approximation to the "true value" of the function $V$.

Let's define $V$ as $V = \mathbb E[f]$, whose discrete approximation is $\hat{V} = \mathbb E[\hat{f}]$ and its Monte Carlo estimate is formulated as $$ \hat{Y} = \frac{1}{N}\sum_{n=1}^{N}\hat{f}^{\,(n)} $$

The expression for the MSE in the aforementioned proof is obtained by doing the following: \begin{eqnarray*} \mathbb E\left[\left(\hat{Y} - V \right)^2 \right]&=&\mathbb E\left[\left(\hat{Y} - \mathbb E\left[\,\hat{f}\,\right] + \mathbb E\left[\,\hat{f}\,\right] - \mathbb E\left[\,f\,\right] \right)^2 \right] = \\ &=& \mathbb E[(\hat{Y} - \mathbb E\left[\,\hat{f}\,\right])^2] + \left(\mathbb E[\hat{f}] - \mathbb E[f] \right)^2 + \mathbb E\left[2\left(\hat{Y} - \mathbb E [\,\hat{f}]\right)\left(\mathbb E\left[\,\hat{f}\,\right] - \mathbb E\left[\,f\,\right]\right) \right] = \\ &=& \frac{1}{N}\mathrm{Var}(\,\hat{f}\,) + \left(\mathbb E[\hat{f}] - \mathbb E[f] \right)^2 \end{eqnarray*}

It is the last line that flusters me. Particularly, my concerns are:

1. I am assuming that the third element of the second line on the right side tends to $0$ since $\mathbb E [\hat{f}] \rightarrow \mathbb E [f]$ and that's why it disappears in the next line, but is this the real reason why this element dissappears?
2. I would think that the first element of the second line, $\mathbb E[(\hat{Y} - \mathbb E\left[\,\hat{f}\,\right])^2]$ is exactly the variance of $\hat{Y}$, so why do we obtain in the last line the variance term multiplied by the factor $\frac{1}{N}$? If we expanded the expression for $\mathrm{Var}(\hat{Y})$, we should obtain the common factor $\frac{1}{N^2}$ by the variance property, and not just $N^{-1}$.

Maybe it is just that I got unnecessarily confused with simple math, so any kind of clarification would be welcome.

Answer 1

I think the notation of the Monte Carlo simulation is awkward for purposes of the proof. Let's simplify and unify the notation a little. This is a 'standard' result in the theory of estimation-- slightly disguised.

MSE equals variance plus squared bias.

You want to estimate the parameter $\tau$ of a distribution that produces observations $T_1, \dots, T_n$ using the sample mean $\bar T = \frac{1}{n}\sum T_i$ as the estimator. The MSE of $\bar T$ as an estimator of $\tau$ is $$E[(\bar T - \tau)^2] = E\{[(\bar T - E(\bar T)]^2\} + [E(\bar T) - \tau]^2 = V(\bar T) - [bias(T)]^2,$$ in which you wish to prove the first equality.

The second equality is by definition of the variance and the bias. One often says that MSE is the variance plus the bias squared. An estimator is called unbiased if its expectation is the parameter to be estimated. Here, $\bar T$ is 'unbiased for $\tau$' if $E(\bar T) = \tau$. The MSE of an unbiased estimator is the same as its variance.

Proof

The proof of the first inequality is much as you wrote it, but I hope less confusing in a more unified notation:

$$E[(\bar T - \tau)^2] = E\{[\bar T - E(\bar T) + E(\bar T) - \tau]^2\} \\ = E\{[\bar T - E(\bar T)]^2\} + [E(\bar T) - \tau]^2 + E\{2[\bar T - E(\bar T)][E(\bar T) - \tau]\}\\ = E\{[\bar T - E(\bar T)]^2\} + [E(\bar T) - \tau]^2\\ + 2[E(\bar T) - E(E(\bar T))]\times [E(\bar T) - \tau],$$

where the term on the last line is $0$ because the factor before $\times$ is $0.$ The last factor is a constant. Finally, there is no harm in observing that $V(\bar T) = \frac{1}{n}V(T_i).$

I will leave it to you to reconcile this version of a standard proof with your original notation.

Answer 2

On your second question, you're correct that the first term is $Var(\hat{Y})$, and you are asking why $Var(\hat{Y})=\frac{1}{N}Var(\hat{f})$?

$$Var(\hat{Y}) = Var\left(\frac{1}{N}\sum_{n=1}^N\hat{f^{(n)}}\right)$$

$$= \frac{1}{N^2}Var\left(\sum_{n=1}^N\hat{f^{(n)}}\right)$$

$$ = \frac{1}{N^2}(N)Var(\hat{f})$$

$$ = \frac{1}{N}Var(\hat{f}).$$