I'm working on a statistics problem:
Given $X = (X_1, X_2, X_3 ... X_n)$ where $X_i \sim N(\mu, \sigma^2)$for i = {1, ... , n} and each of the $X_i$ are i.i.d. Find $E[AX]$ and $Var(AX)$. We're also given: $$A = \begin{pmatrix}1 -\frac{1}{n} & -\frac{1}{n} & -\frac{1}{n} & ...\\-\frac{1}{n} & 1 -\frac{1}{n} & -\frac{1}{n} \\-\frac{1}{n} & -\frac{1}{n} & 1 -\frac{1}{n}\\ ...& & & ... \end{pmatrix}, AX = \begin{pmatrix}X_1 - \bar X \\ X_2 - \bar X \\ X_3 - \bar X\\.\\.\\. \end{pmatrix} $$
We can rewrite A as $A = I - \frac{1}{n}$1 1' Where 1 is a column vector of all 1's.
First $E[AX]$
$$E[AX] = E[\begin{pmatrix}X_1 - \bar X \\ X_2 - \bar X \\ X_3 - \bar X\\.\\.\\. \end{pmatrix}]$$
$$\bar X = \frac{1}{n} \sum_i X_i$$ $$E[\bar X] = \mu$$
Therefore, $$E[AX] = E[\begin{pmatrix}X_1 - \bar X \\ X_2 - \bar X \\ X_3 - \bar X\\.\\.\\. \end{pmatrix}] = 0$$
Then $V(AX)$
Okay here's where I get a little lost:
My original approach is: $$V(AX) = E[(AX - E[AX]) (AX - E[AX])^T] \\= E[(AX - 0) (AX - 0)^T] = E[(A X X^T A^T] = AE[XX^T]A^T$$
And I understand $$E[XX^T] = \begin{pmatrix}E X_1^2 & E X_1X_2 & E X_1X_3 &... \\E X_1X_2 &E X_2^2 & E X_2X_3 \\E X_1X_3 &E X_2X_3 & E X_3^2 \\ ... & & &... \end{pmatrix}, $$
However, approaching it a little differently,
$$Var(AX) = AVar(X)A^T = A \begin{pmatrix}\sigma^2 & 0 & 0 &... \\0 &\sigma^2 & 0 \\0 & 0 & \sigma^2 \\ ... & & &... \end{pmatrix}A^T$$
Where we used the fact that $X_i's$ are all i.i.d, so $Cov(X_i, X_j) = 0$ for $i \neq$ j
Which is what I'm so confused about, both of my approaches seem correct, but they can't be because $$E[XX^T] \neq Var(X)$$
However, from my approaches $$Var(AX) = AE[XX^T]A^T$$ and $$Var(AX) = AVar(X)A^T$$
So are they actually the same? I'm so confused. Any help untangling this confusion is greatly appreciated :-)
Additionally, why doesn't $$Var(AX) = E[XX^T] = \begin{pmatrix}Cov(X_1 - \bar X, X_1 - \bar X) & Cov(X_1 - \bar X, X_2 - \bar X) & Cov(X_1 - \bar X, X_3 - \bar X) &... \\Cov(X_2 - \bar X, X_1 - \bar X) & Cov(X_2 - \bar X, X_2 - \bar X) & Cov(X_2 - \bar X, X_3 - \bar X) \\Cov(X_3 - \bar X, X_1 - \bar X) & Cov(X_3 - \bar X, X_2 - \bar X) & Cov(X_3 - \bar X, X_3 - \bar X) \\ ... & & &... \end{pmatrix} = \begin{pmatrix}\sigma^2 & 0 & 0 &... \\0 &\sigma^2 & 0 \\0 &0 & \sigma^2 \\ ... & & &... \end{pmatrix} = \sigma^2 I$$ the answer?
There are a couple of issues. First, in general $$ V(Y) = E[(Y-EY)(Y-EY)^T] $$ where $Y$ is a random vector. In particular $$ V(AY) = AV(Y)A^T $$ Putting $Y=X$ as in your example we get that $$ V(AX) = AV(X)A^T=\sigma^2\left(I-\frac{1}{n}B\right)\tag{0} $$ where B is a matrix of ones. Note that we should not expect that $V(AX) = \sigma^2I$. Indeed, $$ \begin{align} Var(X_{1}-\bar{X})&=Var\left((1-\frac{1}{n})X_{1}-\frac{1}{n}X_2+\dotsb+-\frac{1}{n}X_{n}\right)\\ &=\left(1-\frac{1}{n}\right)^2\sigma^2+\frac{n-1}{n^2}\sigma^2=\sigma^2\left(1-\frac{1}{n}\right) \end{align} $$ for example by independence. Next in your computation the $X_i$ are not zero mean so the formulas ought to be $EX_i^2 = \sigma^2+\mu^2$ and $EX_{i}X_{j} = \mu^2$. So $$ E[XX^T]=\mu^2B+\sigma^2I $$ But Indeed $$ AE[XX^T]A^T=\mu^2ABA^T+\sigma^2AA^T=\sigma^2AA^T\tag{1} $$ since $AB=0$ the zero matrix. Now we note that $(0)$ and $(1)$ agree.