I want to find the bias and variance of regularized mean estimator which is $$h_{\lambda}(D) \leftarrow \underset{m}{argmin} \ \frac{1}{n} \sum_{i = 1}^N |Y_i - m |^2 + \lambda m^2 $$ $$ h_{\lambda }(D) = \frac{1}{N + \lambda} \sum_{i = 1}^N Y_i$$ $Y$ is a random variable with known distribution and $Y_i$ are independent and $E(Y) = \mu$, $Var(Y) = \sigma^2$ and $D = \{Y_1, Y_2, . . ., Y_n\}$
since bias and variance is defined as $$\mathbb{E}_D \left[ |h(D) - \mu |^2 \right] = \underbrace{|\mathbb{E}_D[h(D)] - \mu|^2}_{Bias} + \underbrace{\mathbb{E}_D \left[|h(D) - \mathbb{E}_D [h(D)]|^2 \right]}_{variance} $$
So my approach is $h_{\lambda}(D) = \bar{Y}$ and $E(\bar{Y}) = \mu$ So bias would be $|\frac{\mu}{\lambda} -\mu|^2 = \frac{\mu^2(1 - \lambda)^2}{\lambda^2}$
but this does not seem correct. What is the correct way of doing it?
Thank you.
$$\frac{\partial}{\partial m}\left(\frac{1}{N} \sum_{i = 1}^N (Y_i - m)^2 + \lambda m^2\right)=-\frac2N\sum_{i = 1}^N (Y_i - m) + 2m\lambda =0$$ $$ -\frac2N\sum_{i = 1}^N Y_i + \frac2N\cdot mN + 2m\lambda =0 $$ $$ 2m(1+\lambda)=\frac2N\sum_{i = 1}^N Y_i $$ $$ m=\frac{1}{1+\lambda}\cdot \bar Y = h_\lambda(D) $$ $$ \mathbb E[h_\lambda(D)] = \frac{1}{1+\lambda} \mathbb E[\bar Y] = \frac{1}{1+\lambda} \mathbb E[Y_1] = \frac{1}{1+\lambda} \mu. $$ So bias is $$ \mathbb E[h_\lambda(D)] - \mu = \frac{1}{1+\lambda} \mu - \mu = -\frac{\lambda\mu}{1+\lambda} $$ Please check the definitions. Bias of an estimate $\theta^*$ of parameter $\theta$ is $\mathbb E[\theta^*]-\theta$, nothing else.
Next, $$ \text{Var}(h_\lambda(D)) = \text{Var}\left(\frac{1}{1+\lambda}\cdot \bar Y\right) = \frac{1}{(1+\lambda)^2} \text{Var}(\bar Y) = \frac{1}{(1+\lambda)^2} \frac{\text{Var}(Y_1)}{N}. $$