What's wrong with my understanding of Liouville's theorem?
A complex lie group is one where group multiplication and inversion are holomorphic. Supposing $G$ is compact and complex, so is $G \times G$. Then multiplication could be viewed as a holomorphic function on a compact set, so it must be constant valued in $G$ by Liouville's. Then since, in particular, the identity times itself is the identity, so is any element times any other element. But this only makes sense if $G=\{e\}$.
I've seen that there is no hesitation in saying that the adjoint representation $\textrm{Ad}: G\to \textrm{Aut}(\mathfrak{g})$ is holomorphic on a compact set and so constant by Liouville in the same context. I assume that the difference must be in that the functions "multiplication" and "inversion" take values in $G$ instead of some other space, but I don't understand what we lose.
Liouville's theorem applies to holomorphic functions from a complex manifold to $\mathbb{C}$: specifically, it says any holomorphic function from a compact complex manifold to $\mathbb{C}$ is locally constant. More generally, it follows that the same conclusion holds for holomorphic functions to $\mathbb{C}^n$, or holomorphic functions to a complex manifold that embeds in $\mathbb{C}^n$. So it applies to $\textrm{Ad}: G\to \textrm{Aut}(\mathfrak{g})$ since picking a basis for $\mathfrak{g}$, $\textrm{Aut}(\mathfrak{g})$ is just $GL_n(\mathbb{C})\subset\mathbb{C}^{n^2}$.
However, Liouville's doesn't typically apply to holomorphic functions $G\to G$, since $G$ typically does not embed in $\mathbb{C}^n$. Indeed, if $G$ did embed in $\mathbb{C}^n$, then Liouville's theorem would apply to the embedding itself and would show that $G$ must be discrete (this works for any compact complex manifold, not just Lie groups).
[Note that even if Liouville's theorem applied, it would only tell you that your map is locally constant, so $G$ is discrete, not necessarily trivial.]