So more or less what I ask in the title: is it possible to identify (uniquely up to bijection) the two-element set in the category of sets as an object that has a particular (categorical) property?
EDIT: Following Hanno's answer, I would like to mention that I know $2$ is a subobject classifier. What I am instead searching for here is some other property that $2$ has (if there is one) and which is not intrinsic to toposes.
On
The two-element set, call it $\Omega$, represents the subset-functor: For any set $X$, there is a natural bijection $$(\ddagger):\qquad\text{Hom}_{\textsf{Set}}(X,\Omega)\ \ \cong\ \ \text{Subset}(X).$$ Because of this, it is called a Subobject Classifier. By the Yoneda-Lemma, an object is determined up to unique isomorphism by the datum of a natural isomorphism $(\ddagger)$.
The two elements $0,1:\ast\to\Omega$ of $\Omega$ can be recovered from $(\ddagger)$ as those corresponding to $\emptyset,\ast\in\text{Subset}(\ast)$.
As the coproduct of the (unique up to canonical isomorphism) one element set with itself perhaps. If we call this two element set $X$, then we have $$X=*\sqcup *,$$ where $*$ is the one-point set.
We can characterise this using the universal property of the coproduct by saying that given maps $f,g:*\rightarrow Y$ for some set $Y$, then there is a unique map, sometimes called $f+g:X\rightarrow Y$, such that for the two inclusion maps $i_1, i_2:*\rightarrow X$, we have $f=(f+g)\circ i_1$, and $g=(f+g)\circ i_2$.