This question is related to Topology of statistical manifolds, but it was never answered to its fullest.
To be concrete, let's just work with the univariate Gaussian distribution $\mathcal{N}(\mu, v)$ (i.e.\ it has mean $\mu$ and variance $v$). In a lot of applied work in information geometry, it seems that the manifold $M$ is implicitly taken to be a $2$-dimensional Riemannian manifold equipped with the Fisher information metric. But if we are to take the definition of a statistical manifold (https://en.wikipedia.org/wiki/Statistical_manifold) seriously, the statistical manifold should be on the space of probability measures, and its mean and variance are just a specific (global) coordinate choice. Thus, when we talk about topologies of the statistical manifold, we shouldn't be talking about the $2$-dimensional manifold (i.e. space of parameters), but rather on the infinite-dimensional space of probability distributions.
Question: I do apologies for my lack of knowledge in information geometry and/or differential geometry. However, I'm sincerely uncertain as to how should one should view the topology of the Gaussian distribution (or really for that matter, any parameterized probability distributions). In particular, is the Gaussian distribution as a statistical manifold "compact" (with respect to what topology)?
The fact that you can consider the space of Gaussian random variables as a submanifold of the space of all probability distributions doesn't really affect it's topology, in the sense that we can simply just view the half-plane as the manifold we're interested in. In differential geometry we typically have to find a set of local coordinate systems, but in the case of the Gaussian statistical manifold it simply admits the one coordinate system. The fact that we equip it with the Fisher metric in information geometry has no bearing on it's topology; the topology was already determined by the fact we picked a single global chart. The manifold itself is diffeomorphic to Euclidean space and so is therefore not compact, and this remains true no matter what metric you attach to it.