Show that $Cov(c_{11}X_1+c_{12}X_2+\dots,+c_{1p}X_p,c_{21}X_1+c_{22}X_2,\dots c_{2p}X_p)$

Question

Show that $$Cov(c_{11}X_1+c_{12}X_2+\dots,+c_{1p}X_p,c_{21}X_1+c_{22}X_2,\dots c_{2p}X_p)=c_1'\Sigma_Xc_2$$ where $c_1'=[c_{11},\dots,c_{1p}]$ and $c_2'=[c_{21},\dots,c_{2p}]$

We have that

$$Cov(c_{11}X_1+c_{12}X_2+\dots,+c_{1p}X_p,c_{21}X_1+c_{22}X_2,\dots c_{2p}X_p)=Cov(\sum_{i=1}^pc_{1i}X_i,\sum_{i=1}^pc_{2i}X_i)$$ I'm not sure but I think that last equality is

$$Cov(\sum_{i=1}^pc_{1i}X_i,\sum_{i=1}^pc_{2i}X_i)=\sum_{i=1}^p\sum_{j=1}^pc_{1i}c_{2i}Cov(X_i,X_j)\textbf{(*)}$$

and

$$c'_1\Sigma_X=\begin{bmatrix}c_{11}&\dots&c_{1p}\end{bmatrix}\times\begin{bmatrix}\sigma_{11}&\dots&\sigma_{1p}\\\sigma_{21}&\dots&\sigma_{2p}\\\dots&\dots&\dots\\\sigma_{p1}&\dots&\sigma_{pp}\end{bmatrix}$$

$$=\begin{bmatrix}\sum_{i=1}^p c_{1i}\sigma_{i1}&\dots,& \sum_{i=1}^p c_{1i}\sigma_{ip}\end{bmatrix}$$

then

$$c_1'\Sigma_X c_2=\begin{bmatrix}\sum_{i=1}^p c_{1i}\sigma_{i1}&\dots,& \sum_{i=1}^p c_{1i}\sigma_{ip}\end{bmatrix}\times \begin{bmatrix}c_{21}\\\dots\\c_{2p}\end{bmatrix}$$

$$=\sum_{i=1}^p c_{1i}c_{21}\sigma_{i1}+\dots +\sum_{i=1}^p c_{ 1i}c_{2p}\sigma_{ip}$$

but I'm not sure if it is equal to (*).

It make any sense ?

Answer 1

$Cov \left( \sum \limits_{i=1}^{p}{c_{1i}X_{i}},\sum\limits_{i=1}^{p}{c_{2i}X_{i}} \right)$

$=E\left[\left(\sum \limits_{i=1}^{p}{c_{1i}X_{i}}\right).\left(\sum \limits_{i=1}^{p}{c_{2i}X_{i}}\right)\right]-E\left[\sum \limits_{i=1}^{p}{c_{1i}X_{i}}\right].E\left[\sum \limits_{i=1}^{p}{c_{2i}X_{i}}\right]$, (by definition of covariance)

$=E[\sum \limits_{i=1}^{p}\sum \limits_{j=1}^{p}{c_{1i}.c_{2j}X_{i}X_{j}}]-\left(\sum \limits_{i=1}^{p}{c_{2i}}E[X_{i}]\right).\left(\sum \limits_{i=1}^{p}{c_{2i}}E[X_{i}]\right)$, by linearity fo expectaion

$=\sum \limits_{i=1}^{p}\sum \limits_{j=1}^{p}{c_{1i}c_{2j}E[X_{i}X_{j}}]-\sum \limits_{i=1}^{p}\sum \limits_{j=1}^{p}{c_{1i}c_{2j}}E[X_{i}]E[X_{j}]$, by linearity fo expectaion

$=\sum \limits_{i=1}^{p}\sum \limits_{j=1}^{p}{c_{1i}c_{2j}(E[X_{i}X_{j}}]-E[X_{i}]E[X_{j}])$

=$ \sum \limits _{i=1}^{p} {c_{1i}.c_{21}.(E[X_iX_1]-E[X_i]E[X_1])} + \sum \limits _{i=1}^{p} {c_{1i}.c_{22}.(E[X_iX_2]-E[X_i]E[X_2])} + \ldots + \sum \limits _{i=1}^{p} {c_{1i}.c_{2p}.(E[X_iX_p]-E[X_i]E[X_p])}$

(expanding the 2nd sum)

$=\begin{bmatrix} \sum \limits _{i=1}^{p} {c_{1i}.(E[X_iX_1]-E[X_i]E[X_1])} & \sum \limits _{i=1}^{p} {c_{1i}.(E[X_iX_2]-E[X_i]E[X_2])} & \ldots & \sum \limits _{i=1}^{p} {c_{1i}.(E[X_iX_p]-E[X_i]E[X_p])} \end{bmatrix}. \begin{bmatrix} c_{21} \\ c_{22} \\ \ldots \\ c_{2p} \end{bmatrix}$ (in matrix form)

$=\begin{bmatrix} c_{11} \\ c_{12} \\ \ldots \\ c_{1p} \end{bmatrix}^T. \begin{bmatrix} E[X_1^2]-E[X_1]^2 & E[X_1X_2]-E[X_1]E[X_2] & \ldots & E[X_1X_p]-E[X_1]E[X_p]\\ E[X_2X_1]-E[X_2]E[X_1] & E[X_2^2]-E[X_2]^2 & \ldots & E[X_2X_p]-E[X_2]E[X_p]\\ \ldots & \ldots & \ldots & \ldots \\ E[X_pX_1]-E[X_p]E[X_1] & E[X_pX_2]-E[X_pX_2] & \ldots & E[X_p^2]-E[X_p]^2 \end{bmatrix}. \begin{bmatrix} c_{21} \\ c_{22} \\ \ldots \\ c_{2p} \end{bmatrix}$

=$c_1^{\prime}.\Sigma_X.c_2$,

where $\Sigma_X(i,j)=E[X_{i}X_{j}]-E[X_{i}]E[X_{j}],\;\forall{i},\forall{j}$