Why does $Var(Y) = Var(\sum_i^n W_iXi) = n(Var(W_i)Var(X_i))$, when $X_i,W_i$ are iid

48 Views Asked by At

This was a step in an explanation of Xavier initialization for neural networks. See http://andyljones.tumblr.com/post/110998971763/an-explanation-of-xavier-initialization (around 3/4 the way down).

It's not clear to me why $Var(Y) = Var(\sum_i^n W_iXi) = n(Var(W_i)Var(X_i))$.

Note: $X_i,W_i$ has mean 0 and are iid.

1

There are 1 best solutions below

0
On BEST ANSWER

$$ Var\left(\sum_{i}W_iX_i\right)=\sum_iVar(W_iX_i)+2\sum_{i<j}Cov\left(W_iX_i,W_jX_j\right)\stackrel{ind}{=}\sum_iVar(W_iX_i)\\=\sum_i\left(\mathbb E[W_i^2X_i^2]-\left(\mathbb E[W_iX_i]\right)^2\right)\stackrel{iid/zero\ mean}{=}nE[X_i^2]E[W_i^2]\stackrel{zero\ mean}{=}n\left(Var(X_i)Var(W_i)\right) $$