I see in papers (here in eq. 3 or here on page 4, for example) that it can be done like this using Fubini $$\mathbb{E}_x\mathbb{E}_\theta\log f(x|\theta)=\mathbb{E}_\theta\mathbb{E}_x\log f(x|\theta)$$ Where $f(x|\theta)$ is a conditional probability density.
I don't immediately see why $\mathbb{E}_\theta\mathbb{E}_x|\log f(x|\theta)|\leq\infty$. Is it actually the case? Or are some additional conditions on $f$ needed?
The identities $$\mathbb{E}_x\mathbb{E}_\theta \bigl[{\bf 1}_{\{f(x|\theta)>1\}}\log f(x|\theta) \bigr]=\mathbb{E}_\theta\mathbb{E}_x\bigl[{\bf 1}_{\{f(x|\theta)>1\}}\log f(x|\theta) \bigr]\quad \; \; (*)$$ and $$\mathbb{E}_x\mathbb{E}_\theta \bigl[{\bf 1}_{\{f(x|\theta)\le 1\}}\log f(x|\theta) \bigr]=\mathbb{E}_\theta\mathbb{E}_x\bigl[{\bf 1}_{\{f(x|\theta)\le 1\}}\log f(x|\theta) \bigr]\quad \; \; (**)$$ always hold by Tonelli's_theorem [1]. Adding them will give the desired formula, provided at least one of them is finite.
On the other hand, if $(*)$ yields $+\infty$ and $(**)$ yields $-\infty$, then the original formula might be undefined. For example, consider the densities $$f(x|\theta)=\frac{C}{|x-\theta |\, [1+(\log |x-\theta|)^2]} \,,$$ where $C$ is chosen to ensure $\int f(x|\theta) \, dx=1$. (One can give an even simpler degenerate example, replacing $\theta$ on the RHS of this formula by $0$.) To verify the example, the integration fact needed is that $$\int_1^\infty \frac{dx}{x (1 +(\log x)^\beta)} $$ is finite for $\beta>1$ but infinite for $\beta=1$. The change of variable $u =\log x$ transforms this to the standard integrals $$\int_0^\infty \frac{du}{ (1 +u^\beta )} \,. $$ Similarly, $$\int_0^1 \frac{dx}{x (1 +|\log x|^\beta )} $$ is finite for $\beta>1$ but infinite for $\beta=1$.
[1] https://en.wikipedia.org/wiki/Fubini%27s_theorem#Tonelli's_theorem_for_non-negative_measurable_functions