Just started Baby Rudin and got struck in this.
While defining order in sets, $<$ was introduced as a relation and for a set to be ordered the condition was: for all $x,y$ belonging to an ordered set $C$, either $x<y$ or $y<x$ or $x=y$.
The first two are alright but how do we define the concept of $=$ in any arbitrary set. I mean $=$ ought to be a relation by itself, shouldn't it?
Absolutely right! In more formal treatments of logic and set theory, the equality relation is often treated as a "God-given" relation that exists on any set. For most texts at the level of baby Rudin say, it is not formally defined, but the informal idea is that a set has been precisely specified only if the meaning of $a = b$ (identity between two elements) in the set has also been specified. For example, once the real numbers have been precisely specified, then an equation like $1 = .999\ldots$ has a definite meaning.
Properties of this equality relation include that $=$ is reflexive, symmetric, and transitive. Many other relations (called equivalence relations) also have these properties; for example, the relation on the set of people "$A$ and $B$ have the same surname" is an equivalence relation, but is not the equality relation.
Perhaps the most important property of equality is the substitution property: that $a = b$ exactly when for any meaningful property or predicate $P$ defined on the set, we have $P(a)$ if and only if $P(b)$. That is, if $a = b$, then the term $b$ can be substituted for a term $a$ in a formula $P(a)$ with no change in the truth value.
All this may seem obvious, and yet -- perhaps unbelievably -- a fuller understanding of the meaning of equality (which has probably been discussed since antiquity) has undergone some important developments within the past few years (in higher-order logic and type theory).