I am currently trying to create a covariance matrix from an adjacency matrix. I would like to create the covariance matrix by the following two steps:
1) Wherever there is a connection, I would place a correlation parameter $\rho$.
2) Replace the diagonals of all $0$'s with a common variance, like $1$.
As a reproducible example, consider the following adjacency matrix for $5$ nodes:
$$ A = \begin{pmatrix} 0 & 0 & 0 & 1 & 1 \\ 0 & 0 & 1 & 0 & 1 \\ 0 & 1 & 0 & 1 & 0 \\ 1 & 0 & 1 & 0 & 0 \\ 1 & 1 & 0 & 0 & 0 \\ \end{pmatrix} $$
We can see from the first row that node $1$ is connected with node $4$ and node $5$.
From this, we replace all the $1$'s with a common correlation parameter $\rho$ and put a $1$ in the diagonal to get:
$$ \Sigma = \begin{pmatrix} 1 & 0 & 0 & \rho & \rho \\ 0 & 1 & \rho & 0 & \rho \\ 0 & \rho & 1 & \rho & 0 \\ \rho & 0 & \rho & 1 & 0 \\ \rho & \rho & 0 & 0 & 1 \\ \end{pmatrix} $$
HOWEVER, this covariance matrix is not positive definite. if I introduce say, $\rho = 0.9$.
Hence, if I fit a multivariate normal model on this covariance matrix, I would get errors in programs. Is there a way around it?
I also noticed if I increase the marginal variances ($1$ above), then the problem goes away. Is there a reason why? Thanks!
If you can reduce the value of $\rho$, it will be positive definite eventually.
From Gershgorin circle theorem, let $R_i= \sum_{j \neq i} |a_{ij}|$, let $D(a_{ii}, R_i)$ be the closed disc centered at $a_{ii}$ with radius $R_i$, each eigenvalue lies in at least one of the disc.
For your particular example $R_i = 2 |\rho|$ and $a_{ii}=1$. If $\rho < \frac12$, then it will be positive definite for sure.