I am watching a lecture on Bayesian inference where the professor says this formula $(*)$ can be derived using the law of total variance but the derivation is not done in the lecture.
https://youtu.be/XtNXQJkgkhI?t=1812
$$var ( \Theta) = var(\hat \Theta) + var(\tilde \Theta) \tag{*}$$
So how can that be done?
Definitions:
a) $\Theta$ is an unknown parameter (r.v.) which we're trying to estimate
b) $\hat \Theta$ is the optimal LMS estimation of that unknown parameter $\hat{\Theta} := \mathbb{E}[\Theta \: | \: X]$
c) $\tilde \Theta$ is the error in the estimation of that unknown parameter i.e. $\tilde \Theta := \hat \Theta - \Theta$
I am looking at the law of total variance and I just don't see (yet) how to prove this formula by using it i.e. I don't see how it's related to $(*)$
https://en.wikipedia.org/wiki/Law_of_total_variance
Any hints/ideas?
It is mentioned in the lecture slides, that $Cov(\hat{\Theta},\tilde{\Theta})=0$, and from this it easily follows, that $$Var(\Theta)=Var(\hat{\Theta}+\tilde{\Theta})=Var(\hat{\Theta})+Var(\tilde{\Theta}) + 2\underbrace{Cov(\hat{\Theta},\tilde{\Theta})}_{=0}.$$ But this doesn't really show the relation to the law of total variance. From the lecture slides $\hat{\Theta}$ is introduced as $$\hat{\Theta} := \mathbb{E}[\Theta \: | \: X],$$ so $$Var(\hat{\Theta}) = Var(\mathbb{E}[\Theta \: | \: X]),$$ and thus the missing piece in order to relate the formula to the law of total variance is to show, that $$Var(\tilde{\Theta}) = \mathbb{E}(Var(\Theta \: | \: X)).$$ I will now argue that this is indeed the case: \begin{align*} Var(\tilde{\Theta})&=Var(-\tilde{\Theta}) \\ &= Var(\Theta - \mathbb{E}[\Theta \: | \: X]) \\ &= \mathbb{E}[(\Theta-\mathbb{E}[\Theta \: | \: X])^2] &\text{(since $\mathbb{E}[\tilde{\Theta}] = 0$)} \\ &= \mathbb{E}[\mathbb{E}[(\Theta-\mathbb{E}[\Theta \: | \: X])^2 \: | \: X]] & \text{(Law of total expectation)} \\ &= \mathbb{E}[Var(\Theta \: | \: X)] &\text{(by definition of conditional variance)} \end{align*}