I am trying to calculate the co-variance of two non-independent variables.
$\sigma$ is a string of length $n$ with bits $\sigma_i$ taking values 1 or -1. One has a p-spin model which is defined by the equation: $$E(\sigma) = \sum_{1 \leq l_1 < l_2 < ... \leq n} J_{l_1,l_2 ... l_p} \sigma_{l_1} \sigma_{l_2} ...\sigma_{l_p} $$ With $ J_{...} $'s being Gaussian distributed. Such that when sampling random states, as $n\to \infty $, the energy is Gaussian distributed with mean 0 and variance 1.
So $$ P(E) = \frac{1}{\sqrt{2\pi}} \mathrm{e}^{-\frac{E^2}{2}} $$ This makes sense to me, since a sum of Gaussian random variables, will, itself be Gaussian distributed. A reference then writes down the joint pdf of two states, which depends on their overlap, given by: $$ q^{(1,2)} = \frac{1}{n} \sum_i \sigma_i^1 \sigma^2_i $$ which takes values between 1 and -1. The joint pdf for sampling two energies given that the bit-flip distance between them is $q$ is given by: $$P(E_1,E_2,q^{(1,2)}) = \left[4\pi^2 (1+q^p) (1-q^p)\right]^{-\frac12} \exp{\left[-\frac{(E_1 + E_2)^2}{4(1+q^p)} - \frac{(E_1 - E_2)^2}{4(1 - q^p)}\right]} $$
I am not sure how I would derive this, but it makes sense. Using Bayes theorem, with this joint pdf, I can calculate the marginal distribution for the likelihood of finding a state with energy $E_1$ given you start at a state with energy $E_0$ and flip a number of bits corresponding to an overlap variable $q_1$, this would be: $$ P(E_1\vert E_0, q_1) = \left[2\pi (1+q_1^p)(1-q_1^p)\right]^{-\frac12}\exp[-\frac{E_1 - q^p E_0}{2(1-q_1^p)(1 + q_1 ^p)}] $$
So, given this conditional distribution, I would like to calculate the co-variance of two values like this. That is, we start at $E$ and apply two sets of flips corresponding to $q^{(1)}$, $q^{(1)}$ to arrive at $E_1$ and $E_2$, which would, themselves, have a bit-flip distance $q^{(1,2)}$.
I would like to calculate the variance of $E_1 - E_2$. Which would be given by $$ \mathrm{Var}(E_1 - E_2) = \mathrm{Var}(E_1) + \mathrm{Var}(E_2) - 2\mathrm{Cov}(E_1, E_2) $$ However, I am having trouble getting my head around how to calculate the covariance. The Variances can simply be read off from the marginal distribution above. I have run some simulations and can guess that the co-variance is $q^{(12)} - q^{(01)}q^{(02)}$, but I cannot show this analytically.
Would someone have some pointers here? Many thanks! I am using a reference: The simplest spin glass, From Mezzard & Gross