I am trying to calcualte the variance of the MCMC estimator shown below (assuming unbiased estimator $E[\theta] = \mu$): $$\theta = \frac{1}{N}\sum_{1}^{n}f(X^{n})$$ However, it is assumed that the samples $X^{n}$ drawn are not independent when $m\neq n$ thus we use $E[f(X^{i})f(X^{j})] - \mu^{2} = Cov(f(X^{i})f(X^{j}))$. My work is the following but I am not sure if it right. $$Var(\theta) = E\left[\frac{1}{N^{2}}\sum_{i}^{N}\sum_{j}^{N}f(X^{i})f(X^{j})\right] - \mu^{2}$$ $$Var(\theta) = E\left[\frac{1}{N^{2}}\sum_{i}^{N}f(X^{i})^{2} + \frac{2}{N^{2}}\sum_{i}^{N-1}\sum_{j=i+1}^{N}f(X^{i})f(X^{j})\right] - \mu^{2}$$ $$Var(\theta) = \frac{Var{f(x)}}{N} + \frac{1}{N^{2}}\sum_{i}^{N}\mu^{2} + \frac{2}{N^{2}}\sum_{i}^{N-1}\sum_{j=i+1}^{N}Cov(f(X^{i})f(X^{j})) + \frac{2}{N^{2}}\sum_{i}^{N-1}\sum_{j=i+1}^{N}\mu_{2} - \mu^{2}$$
I am not studying mathematics so I have expanded the summation by intuition. I checked it for a few cases and it works. Can someone help me please and show me my mistake.
Thank you.
I have figured the answer basically I was missing a few simple expansion steps shown below: $$\frac{1}{N^{2}}\sum_{i=1}^{N}\mu^{2} + \frac{2}{N^{2}}\sum_{i=1}^{N-1}\sum_{j=i+1}^{N}\mu^{2} - \mu^{2}$$ $$\frac{N\mu^{2}}{N^{2}} + \frac{2}{N^2}\sum_{i=1}^{N-1}N\mu^{2}-2\frac{\mu^{2}}{N^{2}}\sum_{i=1}^{N-1}i - \mu^{2} = 0 $$ By using arithmetic sequence and expanding the above gives zero hence from the above formula it is left: $$Var(\theta) = \frac{Var(f(X))}{N} + \frac{2}{N^{2}}\sum_{i=1}^{N-1}\sum_{j=i+1}^{N}Cov(f(X^{i})f(X^{j}))$$