In set theory, we come across the definition of an ordinal as a well-ordered transitive set. And then we show that $\omega$ and all its elements are ordinals.
My question whether there are finite transitive sets which are not ordinals. Also, whether we know all the finite transitive sets.
Yes, there are finite transitive sets which are not ordinals. For example, take $X = \{\emptyset, \{\emptyset\}, \{\{\emptyset\}\}\}$.
You can check quickly that $X$ is transitive, but it is not an ordinal, since it is not linearly ordered under $\in$. We have $\emptyset\not\in \{\{\emptyset\}\}$ and $\{\{\emptyset\}\}\not\in\emptyset$. Of course, the only ordinal of size $3$ is $3 = \{0,1,2\} = \{\emptyset,\{\emptyset\},\{\emptyset,\{\emptyset\}\}\}$.
As for your second question, every set has a transitive closure, and the transitive closure of $X$ is finite if and only if $X$ is hereditarily finite. So the finite transitive sets are exactly the transitive closures of hereditarily finite sets. I don't think you're going to find a more concrete characterization than that, unfortunately.