Given a particular type of large cardinal (Mahlo, Ramsey, Vopenka, etc) is it always the case that the class of such cardinals is closed and unbounded, that is if $\mathcal L\subseteq On$ is a class of large cardinals according to a given type/definition then is it the always the case that:-
1) $\mathcal L$ is a proper class.
2) For all $\alpha\in On$ there is a $\beta\in\mathcal L$ such that $\alpha\le\beta$.
3) For all $A\subseteq\mathcal L$ we have $\sup A\in\mathcal L$.
The reason I ask is because I would like to know if even those large cardinals defined by elementary embeddings like superstrong cardinals also have a purely set-theoretic characterisation.
Thanks
Tl;dr, No, they are not.
Let's leave aside the issue of unboundedness, although this does pose a problem: as Eric Wofsey points out, it is consistent with ZFC that $\mathcal{L}$ is empty for any large cardinal property $\mathcal{L}$ (that's sort of implied by the phrase "large cardinal"), and - for all the properties I'm aware of, at least - existence does not imply frequent existence, in the sense that e.g. it is consistent with ZFC that there are exactly 12 inaccessible cardinals.
We could conceivably get around this by assuming that lots of large cardinals of various types exist. This isn't as trivial as it may sound: for example, if there is a measurable cardinal, then there are infinitely many inaccessible cardinals, and if there is (say) a supercompact cardinal then there are unboundedly many inaccessible cardinals. Let's instead focus on the closed part. It's here that things go very wrong, in what I think is a fatal way: it is in fact essentially never the case that the set of large cardinals of a certain type is closed!
Why? Well, any club contains lots of elements of cofinality $\omega$ (exercise), but no uncountable regular cardinal has cofinality $\omega$; and almost every large cardinal property that I'm aware of demands regularity.
As to your question about defining large cardinals: this is a good suspicion to have! Indeed, on the face of it definitions like "The critical point of some nontrivial elementary embedding $j: V\rightarrow M$" seem unformalizable in the language of set theory. Let me say a few words about this. Note: this isn't really my area, so I hope I don't screw anything up. Inner model theorists, please correct me if necessary!
It turns out that basically all of the large cardinals defined in such a way can be given nice, ZFC-expressible definitions. For example, what I wrote above is the definition of a measurable cardinal, which can also be defined as "A cardinal admitting a countably complete ultrafilter." EDIT: this is wrong - that's the definition of an Ulam measurable cardinal. Obviously these are closed upwards. A cardinal $\kappa$ is measurable if there is a nonprincipal $\kappa$-complete ultrafilter on $\kappa$. It turns out that this approach generalizes: very large large cardinal properties can be defined in terms of either ultrafilters satisfying certain combinatorial properties, or extenders, which are basically sets of ultrafilters satisfying certain fancy properties (see The ABCs of Mice by Schimmerling).
As a taste of this, let me point out that every elementary embedding $j: V\rightarrow M$ yields an ultrafilter $U$ on its critical point $crit(j)=\kappa$ as follows: $$U=\{X: \kappa\in j(X)\}.$$ So this is a way to "collapse" the (class-sized) elementary embedding down to a (set-sized) object. It turns out that this captures a lot of the behavior of $j$, and is very roughly what lets us express this sort of definition in ZFC.