special matrix in terms of its covariance matrix

165 Views Asked by At

How can we find a matrix $S\in \mathcal{M}_{n,n}$ and $Z\in \mathcal{M}_{n,m}$ whose $n$ entries of the $i^{th}$ column $Z_i$ are correlated $Z_i \sim \mathcal{N}(0,S)$ where $S \in \mathcal{M}_{n,n}$ is the covariance matrix of the $n$ entries such that:

$E(Z^T S^{-1}Z)=mI$

1

There are 1 best solutions below

2
On

I'm going to assume that $S$ is already known; otherwise, you can just choose $S=I$ and you're done--except for an $m/n$ issue here that I do not think can be avoided; you'll see below.

Let $\tilde{Z}$ be an $n\times m$ random matrix with elements independently drawn from $\mathcal{N}(0,1)$. $$\textbf{E}(\tilde{Z}_i\tilde{Z}_j^T) = \begin{cases} I & i=j \\ 0_{n,n} & \text{otherwise} \end{cases} \quad\Longrightarrow\quad \tilde{Z}_i\sim\mathcal{N}(0,I).$$ $$\textbf{E}(\tilde{Z}_i^T\tilde{Z}_j) = \begin{cases} n & i=j \\ 0 & \text{otherwise} \end{cases} \quad\Longrightarrow\quad \textbf{E}(\tilde{Z}^T\tilde{Z}) = nI.$$ Incidentally, the second line follows trivially the first, since $\tilde{Z}_i^T\tilde{Z}_j=\mathop{\textrm{Tr}}(\tilde{Z}_j\tilde{Z}_i^T)$.

Now let $L$ be any matrix satisfying $LL^T=S$: e.g., a lower triangular Cholesky factor, or the symmetric square root. Define $Z = L\tilde{Z}$. $$\mathbf{E}(Z_iZ_i^T)=\mathbf{E}(L\tilde{Z}_i\tilde{Z}_i^TL^T)=L\mathbf{E}(\tilde{Z}_i\tilde{Z}_i^T)L^T=LL^T=S\quad\Longrightarrow\quad Z_i\sim\mathcal{N}(0,S).$$ $$\mathbf{E}(Z^TS^{-1}Z)=\mathbf{E}(\tilde{Z}^TL^T(LL^T)^{-1}L\tilde{Z})=\mathbf{E}(\tilde{Z}^TL^TL^{-T}L^{-1}L\tilde{Z})=\mathbf{E}(\tilde{Z}^T\tilde{Z})=nI.$$ We get $nI$, not $mI$ as requested above. I can't help but think this is what is intended.

So in summary: you can construct instances $Z$ satisfying the distribution conditions by multiplying a matrix with i.i.d. elements by a square root of $S$.