Is there a fairly simple object which satisfies the same first order sentences as $\text{ZFC}[\subset] - \{\in\}$? Does the set of measurable subsets of $\mathbb{R}$ with finite measure have this property?
First a word on notation, let a capital Latin letter be a free variable, let a lowercase letter be a bound variable. The closed proper subsets of $\mathbb{R}$ are those that are closed in the standard topology on $\mathbb{R}%$.
Let $\text{ZFC}[\subset]$ be ZFC, but extended with a new relation symbol $\subset$ defined as follows.
$$ A \subset X \;\; \text{if and only if} \;\; \forall a \mathop. a \in A \to a \in X $$
Let $\text{ZFC}[\subset] - \{\in\}$ be a reduct of the extended theory obtained by taking away $\in$.
This object is kind of interesting. I'm wondering if there's anything simpler or more familiar that satisfies the same first order sentences.
$(\mathbb{R_{\ge 0}}, \le)$ is out because $\text{ZFC}[\subset] - \{\in\}$ has finite sets, which have a specific, finite number of subsets ... and counting quantifiers are first order expressible.
$(2^\mathbb{R} \setminus \mathbb{R}, \subset)$ is out because it contains sets like $\mathbb{R} \setminus \{1\}$. $\mathbb{R} \setminus \{1\}$ has a unique, nonempty set $\{1\}$ that is a subset of every set that isn't a subset of $\mathbb{R} \setminus \{1\}$.
The closed proper subsets of $\mathbb{R}$ seem to have some of the right properties. There are finite sets. There's no universal set. However, we can union two closed sets together and get the entire universe.
The subsets of $\mathbb{R}$ with finite measure seem to have more of the right properties. It is possible to union together a countable number of closed subsets of $\mathbb{R}$ with finite measure to give us back $\mathbb{R}$ but I don't think this fact is first-order expressible. Additionally, the relative complement of two sets with finite measure also has finite measure. These subsets together with $\subset$ seems to have a lot of the right properties, but I'm not sure that it really works.
Good question! The theory of your post is studied in this paper of Hamkins under the name "Set-theoretic mereology" (mereology referring to the study of parthood rather than elementhood). The relevant result is Theorem $8$:
Since the algebra of finite-measure sets of reals is an atomic unbounded relatively complemented distributive lattice, your question has an affirmative answer. (Note that this theory is in fact decidable, as Hamkins goes on to say: $\mathsf{ZFC}[\subset]-\{\in\}$ is a profoundly weak theory.)
A quick coda on the phrasing of results:
Hamkins talks about set-theoretic mereology as the theory of the class-sized structure $(V,\subseteq)$. This is on the face of things quite different from the theory of the OP, and initially we shouldn't even expect $Th(V,\subseteq)$ to be definable in $V$ since we can't in general define truth predicates for class-sized structures (such as $Th(V,\in)$, per Tarski). So it is a nontrivial consequence of Theorem $8$ that $\mathsf{ZFC}$ does capture all the "subset facts" about the universe, that is, that your $\mathsf{ZFC}[\subset]\setminus\{\in\}$ is exactly $Th(V,\subseteq)$.