Given a sufficiently nice PDF $f:M\times \Theta \to \mathbb{R}$ where $M\subset \mathbb{R}^D$ and $\Theta\subset \mathbb{R}^p$ if the Fisher Information matrix $$I_{ij}(\vec{\theta})=\mathbb{E}\left[\left(\partial_{\theta_i} \log f(\vec{X}, \vec{\theta})\right) \cdot \left(\partial_{\theta_j} \log f(\vec{X}, \vec{\theta})\right)\bigg|\theta\right]$$ is positive definite (PD) then it forms a metric tensor in the variable $\theta$ and we can say that $(\Theta, I)$ is a manifold, called a statistical manifold. Thus, for nice enough PDFs, we can always find an associated a statistical manifold.
Question: Given a Riemannian manifold $(\Theta,g)$ with metric tensor $g$, can we find a PDF $f:M\times \Theta\to\mathbb{R}$ (for some $M$) such that $I(\vec{\theta})=g(\vec{\theta})$? If so, is such a PDF unique? In other words: is every metric tensor the Fisher information metric for some RV $X$? Another way to ask is whether every metric tensor $g$ on $\Theta$ can be written as $$g(\theta) = - \mathbb{E}\left[\nabla_\theta^2 \log f(\vec{X}, \vec{\theta})\bigg| \theta\right],$$ for some sufficiently nice $f$?
Some thoughts: I would wager that the class of manifolds is larger than the class of statistical manifolds (those whose metric tensors are the FIM of some RV). But this is just a rough guess and I am not sure what tools to use to approach this question. If anybody has suggestions or hints I would gladly expend more effort on this and add some attempts.
Statistical manifolds were introduced by Lauritzen in this work (even if they were already known by people working in information geometry). The idea behind is precisely the one mentioned by @Didier in their comment. Note, however, that Lauritzen uses a symmetric, 3-covariant tensor called the skewness tensor $T$ instead of the dually related connections mentioned in the comment.
Then, Hong Van Le proved here that every compact $C^{1}$-statistical manifold $(M,g,T)$ is actually a statistical model (see also here and here). This means that you can represent points in $M$ as probability distributions on a given outcome space $\mathcal{X}$ in such a way that $g$ coincides with the Fisher-Rao metric tensor and $T$ with the so-called Amari-Cencov tensor coming from realizing $M$ inside $\mathcal{P}(\mathcal{X})$ (probability distributions on $\mathcal{X}$).
Unfortunately, I do not remember the details of the proof, nor I remember if meaningful comments are made concerning the non-compact case.