Consider a random vector $X$ defined on the probability space $(\Omega, \mathcal{F}, \mathbb{P})$, $X: \Omega \rightarrow \mathbb{R}^k$. Suppose $X$ has probability density $p_{\theta_0}$ with respect to a probability measure $\mu$ on the measurable space $(\mathbb{R}^k, \mathcal{B}(\mathbb{R}^k))$, where $\theta_0 \in \Theta \subseteq \mathbb{R}^l$ is a parameter and $\mathcal{B}(\mathbb{R}^k)$ is the Borel $\sigma$-algebra on $\mathbb{R}^k$. Suppose $X$ has probability distribution function $P_{\theta_0}$.
Consider the Fisher information matrix
$$ I(\theta_0):=E_{P_{\theta_0}}\left(\dot{l}_{\theta_0}(X)\dot{l}_{\theta_0}(X)^T\right) $$ where $\dot{l}_{\theta_0}(X)$ is the derivative of the map $\theta \rightarrow \log(p_{\theta}(X))$ with respect to $\theta$ evaluated at $\theta_0$.
Question: Suppose we are told that $I(\theta_0)$ exists and is non-singular. Does this statement require assumptions on the finiteness of the elements in $I(\theta_0)$?
My thoughts and doubts: existence of $I(\theta_0)$ just means that all Lebesgue integrals are well-defined (but they could be $\infty, -\infty$); hence it shouldn't have implications for the finiteness of the elements in $I(\theta_0)$; non-singularity means that the determinant of $I(\theta_0)$, $\det(I(\theta_0))$, is non-zero; stating that $\det(I(\theta_0))\neq 0$ implies that we should be able to compute $\det(I(\theta_0))$; hence, I'm wondering whether this requires assuming that all elements in $I(\theta_0)$ are finite in order to avoid indeterminate forms such as $\infty-\infty$, $0*\infty$, etc.