I was looking at the proof for optimal weighting matrix when using GMM (slide 35 here) At one point, they take the variance of both sides of following expression (where Z is a random vector)
$A_1Z = A_0Z + (A_1-A_0)Z$
And get the following:
$var(A_1 Z) =var( A_0Z) + var((A_1-A_0)Z) + A_0 var(Z)(A_1'-A_0') + (A_1-A_0)Var(Z)A_0'$
which evalates to $var( A_0Z) + var((A_1-A_0)Z)$
After a lot of thinking, I don't seem to get how the term $var(Z)(A_1'-A_0') + (A_1-A_0)Var(Z)A_0'$ shows up. Can anyone help please, what part of the linear algebra class I missed?
In your case the random vector $Z$ has zero mean. So your calculation simplifies to $\operatorname{Var}(X+Y)$ where $X:=A_0 Z$ and $Y:=(A_1-A_0)Z$ are mean-zero random vectors. Since $X+Y$ will also have mean zero, its variance is $\operatorname{Var}(X+Y)=E(X+Y)(X+Y)'$, which equals $$ E(XX' + YY' +XY' + YX')=\operatorname{Var}(X)+\operatorname{Var}(Y)+E(XY')+E(YX'), $$ where the final two terms are $$E(XY')=E\left(A_0Z[(A_1-A_0)Z]'\right)=A_0E(ZZ')(A_1'-A_0')$$ and $$E(YX')=E\left([(A_1-A_0)Z](A_0Z)'\right)=(A_1-A_0)E(ZZ')A_0'.$$ To finish off, observe that $E(ZZ')=\operatorname{Var}(Z)$ since $Z$ has zero mean.