Suppose that $f(\theta_1,\dots,\theta_n,x)$ is a (positive) probability density function over a finite set $\mathbb{X}$ defined for an open subset $\Theta\in\mathbb{R}^n$. That is, for every $\theta\in\Theta$ and $x\in\mathbb{X}$, $f(\theta_1,\dots,\theta_n;x)>0$ and $$ \sum_x f(\theta_1,\dots,\theta_n;x) = 1. $$ Let \begin{equation} y_{i,j}(\theta) = \sum_x f(\theta_1,\dots,\theta_n;;x)\frac{\partial}{\partial\theta_i}\log f(\theta_1,\dots,\theta_n;x) \frac{\partial}{\partial\theta_j}\log f(\theta_1,\dots,\theta_n;x). \end{equation} (This is called Fisher information in statistics)
Let $$\widehat{\eta_i}(x) := g_i(\theta_1,\dots,\theta_n;x)$$ and
$$\eta_i := \sum_x f(\theta_1,\dots,\theta_n;x) g_i(\theta_1,\dots,\theta_n;x),$$ for some "nice enough" functions $g_i$. Hence $\eta$ is a function of $\theta$.
It can also be shown that $$ \frac{\partial}{\partial\theta_i}\log f(\theta_1,\dots,\theta_n;x) = \widehat{\eta_i}(x) - \eta_i. $$ By a calculation I am getting $$ \frac{\partial\eta_i}{\partial\theta_j} = y_{i,j}(\theta) + k \cdot \eta_i\eta_j, $$ for some real constant $k$.
However, what I would like to have is $$ \frac{\partial\eta_i}{\partial\theta_j} = y_{i,j}(\theta). $$ So is it possible to modify the $g_i$'s (by scaling/shifting the existing $g_i$) appropriately so that we get the above equation? I tried a bit, couldn't succeed. Any help is greatly appreciated.
Given fixed $i\in \{1,\dots, n\}$ we are looking for a function $\eta_i$ such that
$$\nabla\cdot\eta_i=\left(\frac{\partial\eta_i}{\partial\theta_1},\dots, \frac{\partial\eta_i}{\partial\theta_n}\right)=(y_{i,1},\dots, y_{i,n})=y_i,$$
that is when $y_i$ is a conservative vector field.
I expect this holds when $\Theta$ is simply connected and the functions $y_{i,j}$ are sufficiently smooth (I expect, continuously differentiable). I tried to make this claim more explicit and find a (rigorous) proof for it, but I failed.
Indeed, (assuming $\log=\log_e$) we have $$y_{i,j}=\sum_x \frac 1f\frac{\partial f}{\partial\theta_i}\frac{\partial f}{\partial\theta_j}.$$ Then for each $k\ne j$ we have
$$\frac{\partial y_{i,j}}{\partial\theta_k}=\sum_x -\frac 1{f^2}\frac{\partial f}{\partial\theta_k}\frac{\partial f}{\partial\theta_i}\frac{\partial f}{\partial\theta_j}+ \frac 1f\frac{\partial^2 f}{\partial\theta_i\partial\theta_k}\frac{\partial f}{\partial\theta_j}+ \frac 1f\frac{\partial f}{\partial\theta_j}\frac{\partial^2 f}{\partial\theta_j\partial\theta_k}.$$
If all second order partial derivatives of $f$ with respect to $\theta$’s are continuous then by Schwarz's theorem or Clairaut’s_theorem on equality of mixed partials we have
$\frac{\partial^2 f}{\partial\theta_i\partial\theta_k}=\frac{\partial^2 f}{\partial\theta_k\partial\theta_i}$ and $\frac{\partial^2 f}{\partial\theta_j\partial\theta_k}=\frac{\partial^2 f}{\partial\theta_k\partial\theta_j}$, so $\frac{\partial y_{i,j}}{\partial\theta_k}=\frac{\partial y_{i,k}}{\partial\theta_j}$.
These conditions should be necessarily for $y_i$ to be a conservative vector field (see, for instance, this and this questions).
Moreover, if $\Theta$ is simply connected and the functions $y_{i,j}$ are sufficiently smooth this should be also a sufficient condition. I tried to find an explicit reference for this claim, but I failed beacuse of the following two reasons.
Classical multivariable calculus books which I found deal with at most three dimensional case (see the references, in particular, [J, Theorem 6] when $n=2$ and $\Theta$ is a domain, [MI, Theorem 10.2] and [G, Theorem 1.3] when $n=3$), whereas a general case is considered by means of differential forms (see the end of Chapter Irrotational vector fields of the article from Wikipedia and [GW], Sectiton 3 “Differential forms” and Section 5 “The moral of the story”), which, I guess, are proved using Generalizations of Gradient theorem.
I am not satisfied with the clarity of formulations and rigor of the proofs from the above references. I found better references from this point of view for continuously differentiable vector fields for $n$ equal to $2$ and $3$ in the books [F, Ch. 15, $\S$ 3] and [LEB, p.340-341], which I studied when I was a student, but they are in Russian and Ukrainian. So I hope other MSE users can provide to you good English references.
On the other hand, procedures to construct potential of a vector field for $n\le 3$ are known, see, for instance, [G, Procedure 1.4], [MI, Section “Finding the Potential, If Possible”], [S, 25.2 Finding Potential Functions], [GW, Subsection “Relating Gradients and Curls” for $n=3$, closed $\Theta$ and continuously differentiable $y_i$].
References
[F] G. Fichtenholz, Differential and Integral Calculus, vol. III, 4-th edition, M.: Nauka, 1968, (in Russian).
[G] R. Garrett, Potential Functions and Conservative Vector Fields.
[GW] E.H. Goins, T.M. Washington, A Tasty Combination: Multivariable Calculus and Differential Forms.
[J] D. Joyce, Conservative vector fields in Math 131 Multivariate Calculus in Clark University, Spring 2014.
[LEB] I. Lyashko, V. Emel’yanov, O. Boyarchuk, Mathematical analysis, vol. 2, Kyiv, Vyshcha shkola, 1993 (in Ukrainian).
[MI] Multidimensional Integration.
[S] T. Silber, Multivariable Calculus Study Guide.