If the statement "Is an element of" is a primitive term (is not defined/you cant determine its truth) then how do you determine the truth of statements such as "If $x$ is in $S$, then $x$ has property P"? or "Some $x$ in $S$ has the property P"?
$(\forall x)\left ( x\in S \Rightarrow P(x) \right )$ or $\left ( \exists x \right )(x\in S\, \wedge\, P(x) )$
I am aware of the axiom of specification, where you can define the properties of the elements and use that property to determine what belongs to the set. However isn't that exactly the same as saying,
$(\forall x)(x\in S\equiv P(x))$?
And is this not what leads to Russell's paradox? Just take P(x) as $\sim (x\in x)$ and you get
$(\forall x)(x\in S \equiv \sim (x\in x)) $
Then by universal instantiation you get the inconsistent sentence
$(s\in S \equiv \sim (s\in s)) $
So it seems that "x is s" cannot be interpreted as a sentence and if that is the case how can we interpret truth of quantified sentences in set theory? Let alone prove anything in the foundations of mathematics??
Apart from logical tautologies, you can't determine the truth of anything by logic before you choose a collection of axioms to reason from. The axioms you choose will function as (partial) definitions of the primitive notions in your language -- in the sense that choosing such-and-such axioms amounts to declaring "I'm only interested in your concept of $\in$ if it satisfies such-and-such; otherwise you're talking about something different than I am."
You can then hope that there actually is something out there that behaves like your axioms promise that the primitive notions will behave. But ultimately that is something you need to take on faith; logic in itself will not prove it for you.
(That is, if your context is that you're using logic in an attempt to build a foundation for mathematics. If you're just studying logic as an interesting mathematical theory among other things you can do with mathematics, you will already have access to all of the usual machinery for reasoning about things, including sets and so forth, and then it might be possible for you to prove that a particular theory has a model).
Yes, allowing sets to be defined by properties willy-nilly is exactly what leads to Russell's paradox, and your presentation of it is mostly standard.
What this ought to tell you is that you had better not choose to use a collection of axioms that includes $$ (\exists s)(\forall x)(x\in s\Leftrightarrow P(x)) $$ for every formula $P$ that doesn't contain the variable name $s$.
That's not a logical problem, though -- you can still (we hope!) do set theory as long as you're more careful with choosing your axioms such that they don't include the ones that leads to Russell's paradox.