I'm reading Sompolinsky et al's 'Chaos in Random Neural Networks' paper(pdf is behind a paywall) but I will try my best to distill the relevant information.
The paper explores how to create, and the properties of, chaotic recurrent neural networks that produce spontaneous yet non-exploding dynamics.
The network consists of. N continuous variables ('neurons') $\{S_i(t)\}_{i=1}^N$ with $-1\leq S_i \leq 1$. Associated with each neuron is a local field $h_i$, with $-\infty<h_i \leq +\infty$ and is defined through the relationship $S_i(t)=\phi(h_i(t))$ is a nonlinear gain function.
The gain function is assumed to 1) have a sigmoid shape $\phi(\pm \infty)$ and $\phi(-x) = -\phi(x)$. For concreteness, the authors use $$\phi(x) = \tanh(gx) \hspace{5mm} (1)$$ where $g$ 'measures the nonlinearity' of the gain function(the higher the value of g, the more 'bendy' and less like a straight line, the sigmoid becomes as it approaches its asymptotes more rapidly.)
The network dynamics are defined by N coupled first-order differential equations:
$$\dot{h}_i(t) = -h_i(t) + \sum^N_{j=1}J_{ij}\phi(h_j) \hspace{10mm} (2)$$
J_ij couples the output of the $j$th neuron to the input of the $i$th neuron and $J_{ii}=0$.
The authors assume that the network synaptic weights (the $J_{ij}s$ are independent zero-mean Gaussian random variables each with variance $[J^2_{ij}]_J = J^2/N$ so that the intensive parameter in equation (1) is $gJ$.
In general the long-term behaviour of this system will depend on the specific values of the $J_{ij}$ but for large N, mean-field theory can be used to show that a well defined typical long-time behaviour exists and that this theory is exact in the limit $ N \to \infty$.
They claim that according to the mean-field theory, equation (2) can be reduced to $$\dot{h}_i(t) = -h_i(t) + \eta_i(t) \hspace{10mm} (3)$$ where:
- $\eta_i$ is a time-dependent Gaussian field generated by the random inputs to neuron i from the other neurons - i.e. the last term in equation (2).
I.e. $$\sum^N_{j=1}J_{ij}\phi(h_j(t)) \to \eta_i(t)$$, assuming enough time has passed,the mean-field theory and the distributional assumptions made on the weights. To be precise in terms of the time assumption, we're just assuming that enough time has passed so that the system has reached a steady state so that correlations depend only on time separations.
Thus, the mean of $\eta_i(t)$ is zero and the second moment is determined self-consistently from equation (2), yielding $$[\eta_i(t)\eta_i(t+\tau)]_{\mathbb{N}_{\eta}} = J^2C(\tau) \hspace{10mm} (4)$$ where $\mathbb{N}_{\eta}$ denotes the Gaussian distribution of $\eta$.
They then go on to study the local-field autocorrelation $$\delta(\tau) = [h_i(t)h_i(t+\tau)]_{\mathbb{N}_{\eta}} \hspace{10mm} (5) $$ which by equations (3) and (4) obeys $\delta - \ddot{\delta}=J^2C$ .
Here's where I get lost: 'equations (4) and (5) can be reduced to the following equation for $\delta$
$$\ddot{\delta} = -\partial{V}/\partial{\delta} \hspace{10mm} (6)$$
$$V(\delta) = -\frac{1}{2}\delta^2 + \int^{+ \infty}_{- \infty} Dz \left(\int^{+\infty}_{-\infty} Dx \, \Phi((\delta(0) - |\delta|^{\frac{1}{2}}z)\right)^2 \hspace{10mm} (7)$$ where $Dz=dz\exp(-z^2/2)/(2\pi)^{\frac{1}{2}}$, and in general $\Phi(x) = \int^x_0dy\phi(y)$. In the particular example of (1) $\Phi(x) = (gJ)^{-1}\ln\cosh(gJx)."
they also go on to say that equation (6) can be viewed as a one-dimensional motion of $\delta(t)$ under the Newtonian potential $V(\delta)$.
How do equations (6) and (7) follow from equations (4) and (5)? Is this a well known relationship or some sort of canonical form? Where can I learn more about the details for this?