As is well known, Reinhardt cardinals are inconsistent with $\mathsf{ZFC}$, but many of the proofs I've seen of this rely on combinatorial or club/stationary set properties.
If there's a (somewhat easy to see and non-trivial) first-order way of stating "$\kappa$ is Reinhardt", then we could easily appeal to elementarity to disprove their existence: the least Reinhardt $\kappa$ would have $V\models\text{“}\kappa\text{ is the least Reinhardt}\text{"}$ and so $V\models\text{“}j(\kappa)\text{ is the least Reinhardt}\text{"}$.
Does anyone know of such a definition, or are we more-or-less forced into the usual routes?
No, there is not (well, "$0=1$" ... :P).
You're quite right that from a first-order definition of Reinhardt-ness, we would conclude immediately the nonexistence of Reinhardt cardinals. Indeed, more can be said: Suzuki proved (doi: 10.2307/2586799)that there is no definable-from-parameters elementary embedding from $V$ into $V$. This is a harder result: in principle, the relevant parameters might be moved by the embedding, so you don't get the obvious contradiction. (EDIT: note that Suzuki did not assume that $V$ satisfies choice.)
There is, however, a "first-order version" of Reinhardt cardinals: namely, we look at nontrivial elementary embeddings of $V_{\lambda+2}$ into $V_{\lambda +2}$ for some $\lambda$. These are more-or-less equivalent to Reinhardt cardinals for our purposes here:
Every Reinhardt cardinal yields such an embedding: if $j:V\rightarrow V$ is nontrivial elementary with critical point $\kappa$, let $$\lambda=\sup_{n\in\omega}(j^{(n)}(\kappa)).$$ Then $j$ restricts to an elementary embedding $\hat{j}:V_{\lambda+2}\rightarrow V_{\lambda+2}$ with critical point $\kappa$.
Conversely, Kunen's analysis applies to these without changes: assuming ZFC + $\kappa$ is Reinhardt, the crucial combinatorial object in the Kunen inconsistency (namely, the appropriate Jonsson function) exists in $V_{\lambda+2}$. That is, Kunen's argument without substantial changes shows that ZFC proves "There is no nontrivial elementary embedding from $V_{\lambda+2}$ to $V_{\lambda+2}$ for any $\lambda$."
EDIT: Alessandro Codenotti reminded me in a comment below that there is actually a small subtlety here - this is one of those times where the choice of pairing function actually matters. You need to use a flat pairing function here. If you use a non-flat pairing function, you wind up getting a weaker result - e.g. at a glance the usual Kuratowski pairing function gives only that there are no nontrivial self-elementary embeddings of $V_{\lambda+\color{red}{4}}$.
Interestingly, the above "localization" suggests a couple weakenings of the Reinhardt principle which turn out to be quite interesting: nontrivial elementary embeddings from $V_{\lambda+1}$ to $V_{\lambda+1}$, or from $V_\lambda$ to $V_\lambda$, for some $\lambda$. These are "small enough" (if just barely) to escape Kunen's argument, and are called $I_1$ and $I_3$ embeddings respectively. There are also other less obvious variations; collectively, these are called rank-into-rank cardinals.