I was thinking about the two element set today, and how it can be used to represent a binary function. One very important binary functions is whether or not an element in set $A$ is in another set $B$. A map from $A$ to $2$ could define membership in set $B$. So, the two element set $2$, is important for membership. How are other sets used in the semantics of set theory itself? What is the abstract theory of this phenomenon?
A commenter has pointed out that I am talking about subobject classifiers. These are based on the limit of the cospan diagram $X \rightarrow 2$ and $1 \rightarrow 2$. Is there a heirarchy of "internal concepts" that we can construct with sets of increasing cardinality using composition of cospans and their limits?
The abstract theory of this phenomenon deals with representable functors.
Let $\mathbf{Set}$ denote the category of sets. Given a category $C$ and a functor $F\colon C^{\mathrm op}\to \mathbf{Set}$ we can ask if it is representable, i.e. whether it is naturally isomorphic to a functor $H_a=\operatorname{Hom}(-,a)$ for an object $a$ in $C$.
In your example, $F$ is the functor $\mathbf{Set}\to\mathbf{Set}$ which sends a set to its powerset. This functor is representable by the $2$-element set.
In general, we think of a functor $F\colon C^{\mathrm op}\to \mathbf{Set}$ as assigning to every object $c\in C$ a set of "structures $S$ of a certain type on $c$". Being representable by an object $a\in C$ then means that there is a "universal structure $S_{\mathrm{univ}}$ of that type on $a$" in the sense that every other structure $S$ on $c$ arises as the pullback of $S_{\mathrm{univ}}$ along a unique morphism $c\to a$.
Let me give a more non-trivial example: Fix sets $x,y$ and consider the functor $\mathrm{Set}\to\mathrm{Set}$ given by $c\mapsto \operatorname{Hom}(c\times x,y)$. This functor represented by the set $y^x$ equipped with the universal element of $\operatorname{Hom}(y^x\times x, y)$ given by the evaluation map $y^x\times x\to y$.
Of course one can also dualize and consider functors $C\to \mathrm{Set}$ which are corepresentable (i.e. isomorphic to $\operatorname{Hom}(a,-)$). A very basic example would be the identity $\mathrm{Set}\to\mathrm{Set}$ which is corepresented by any singleton set.