I have heard it said that set theory can be seen as the foundation for the rest of mathematics. The article on set theory at brilliant.org put it as:
Set theory is important mainly because it serves as a foundation for the rest of mathematics -- it provides the axioms from which the rest of mathematics is built up.
I also saw an answer already on math.stackexchange.com that also further buttresses the point.
But the part I have failed to find is an example of such approach of using Set theory to formalize an other aspect of Mathematics.
For example, how would algebra or geometry or logic be expressed using set theory?
To answer the last part of your question in detail ("how would algebra or geometry or logic be expressed using set theory?") would take way longer than a stackexchange answer. Very briefly, and in reverse order, and just for geometry:
If you want to see some of this fleshed out, the book by Enderton, Elements of Set Theory, is one reference.
I sketched how to grow a small part of the vast eco-system of modern math using just sets as the basic "biochemistry". Of course you'll also want to prove things about your mathematical objects, be they points, or vector spaces, or what have you. For this we need an axiom system for sets. Zermelo-Fraenkel set theory (ZFC) is a popular choice.
Now for the plot twist. This account of how set theory is "the foundation for mathematics" is kind of old-fashioned, and not everyone takes it that seriously any more.
It's not that there's anything wrong with the constructions (1)-(6). They're still perfectly valid. But if (say) you want to study euclidean geometry, only step (1) (also known as analytic geometry) helps you in any way. If you want to do real analysis, you can start by assuming the real numbers constitute a complete ordered field, and utterly ignore the possibility that a real number isn't just "a point", but has internal structure as (say) a Dedekind cut. Etc. And truth be told, some of the "encoding tricks" of (2)-(6) smack of contrivance, clever as they are.
Axiomatic set theory is a fascinating and beautiful subject in its own right. Also, the language of sets (and functions) has taken over math almost completely. Take something like Galois theory. If you look at pre-20th century treatments, the subject looks very different: all talk about rational expressions, permutations, and equations. Starting in the 20th century the vocabulary and conceptual framework changes: sets with structure, chiefly fields and automorphism groups. (Not that the rational expressions and permutations have disappeared!) It's a similar story for nearly every other branch of math, at least if the branch has pre-20th roots. Nowadays differential geometry is all about manifolds, but for Gauss it was about differential equations and surfaces in a more naive sense.
Lately (the past 40 years?) math has seen another framework shift, towards category theory. But that's another topic.