Weighted variance of a small sample

707 Views Asked by At

I am trying to calculate the variance of a small sample. I have the data:

    PSU STRATUM WEIGHTS COUNTS
1   1   1          2    1
2   2   1          1    1
3   3   1          1    1
4   3   1          1    1
5   2   1          1    1
6   2   1          1    1
7   1   1          1    1
8   1   1          1    2

I have the variance calculated from R with the output:

       variance     SE
COUNTS  0.11287 0.0829

The mean that I calculated was $\mu=1.1111$. To compute the variance I used the formula

$$\sigma_{weighted}^2 = \dfrac{\sum_{i=1}^Nw_i(x_i-\mu)^2}{\sum_{i=1}^N w_i} $$ The answer I got from this was 0.09876543.

Any help or comments would be greatly appreciated.

$\bf{UPDATE:}$ I found an article for a weighted variance given by

$$ Var(\bar{x}_w) = \dfrac{\sum_{i=1}^Nw_i^2}{(\sum_{i=1}^Nw_i)^2}\sigma_{weighted}$$

The answer I got with this was 0.1207133 which is still off.

1

There are 1 best solutions below

2
On BEST ANSWER

Your answer calculated the biased weighted variance, and I think you are probably more interested in the unbiased weighted variance, which is a bit trickier to calculate. The formula is: $$ s^2_{\textrm{weighted}} = \frac{\sum_{i=1}^N w_i}{\left(\sum_{i=1}^N w_i\right)^2 - \sum_{i=1}^N w_i^2}\cdot\sum_{i=1}^N w_i \left(x_i - \mu\right)^2 $$ where $$ \mu = \frac{\sum_{i=1}^N w_i x_i}{\sum_{i=1}^N w_i} $$

In your example, the result should be: $$\frac{9}{9^2 - 11}\cdot \frac{8}{9} = \frac{4}{35}\approx 0.114285714285714$$