How does one define genus of curves defined over finite fields (or over general fields)? I only know the geometric definition for smooth plane curves $g = \frac{(d-1)(d-2)}{2}$, and the definition for curves over $\mathbb{C}$ as the number of linearly independent holomorphic $1$-forms over the curve.
How does one define this concept in general? What's the analogous of "holomorphic" forms in finite fields?
For a smooth curve $X$ over a field $k$ (curve = integral scheme with structure morphism separated of finite type) with $H^0(X,\mathcal{O}_X)=k$, the usual definition of the geometric genus is $$p_g(X)=\dim_k H^0(X,\Omega_{X/k})$$ where $\Omega_{X/k}$ is the sheaf of Kalher differentials associated to the morphism $X\to \operatorname{Spec} k$. This exactly recovers the concept of linearly independent holomorphic differential forms you mention in your post, except we say "regular" instead of holomorphic.
There's also the arithmetic genus, which for a general projective scheme $X$ over a field $k$ is defined as $$p_a(X) = (-1)^{\dim X}(\chi(\mathcal{O}_X)-1).$$ If $X$ satisfies the same assumptions as in the first paragraph, this becomes $1-\chi(\mathcal{O}_X)=\dim H^1(X,\mathcal{O}_X)$ which equals the geometric genus by Serre duality.
For more discussion on replacing "holomorphic" by "regular", check almost any algebraic geometry book that does not work over $\Bbb C$ the whole time: Vakil, Hartshorne, etc. These books will give you the whole story in a more digestible format than an MSE answer; the short version is that everything comes out of the coordinate algebras of affine charts and things really flow rather nicely once you get in to it.