From a set-oriented point of view, a relation between sets $A$ and $B$ is a subset $\rho \subset A \times B$ of their product. By imposing further axioms (e.g. left-totality, right-totality, symmetry, transitivity, etc.) we may produce a number of useful concepts: (partial) functions, equivalence relations, (partial) orders, etc.
- $\rho$ is a function if and only if it is left-total and right-unique.
- $\rho$ is a partial function if and only if it is right-unique.
- $\rho$ is an equivalence if and only if it is reflexive, symmetric, transitive...
What I find interesting is how this process generalizes to contexts other than sets. Consider groups. Let $A$ and $B$ be groups, then a subgroup $\rho \leq A \times B$ (direct sum) may be called a relation of groups. Observe how
$\rho$ is a group homomorphism if and only if it is left-total and right-unique.
Explanation: Since $\rho$ is a subgroup of $A \times B$, we have $(1, 1) \in \rho$; in other words, $1\rho = 1$. If $x\rho = x'$ and $y\rho = y'$, i.e. $(x, x'), (y, y') \in \rho$, then $(xy, x'y')$, i.e. $(xy)\rho = (x\rho)(y\rho) = x'y'$. And if $x\rho = x'$ or $(x, x') \in \rho$, then also $(x^{-1}, x'{}^{-1}) \in \rho$, so $x^{-1}\rho = (x\rho)^{-1} = x'{}^{-1}$.
$\rho$ is a partial group homomorphism if and only if it is right-unique.
$\rho$ is a congruence relation if and only if it is reflexive, symmetric, transitive.
I am quite sure that this works for $G$-sets, rings, $R$-modules, and others. So this is quite beautiful; we only need a notion of a subobject and of a product and then we get the rest (such as homomorphisms, orders, congruences) for free. And category theory can provide us with both.
I am interested in whether there is a general notion of a relation or "relation object" in category theory and how one further derives the notions above from this. Or maybe category theory has a completely different approach to defining these. What is this approach? I am particularly interested in how the process described above extends to more general categories (say, Cartesian closed categories).
A good place to start is with the notion of a regular category. You can push things further than this, but regular categories are a nice balance of common enough to be useful in many situations but stringent enough to let use say interesting things in those situations.
For instance, every category of models for some algebraic theory is regular (this includes the categories of $G$-sets, rings, $R$-modules, etc. that you've noticed, and is some evidence that regular categories are "common enough"). As for the "interesting things" we can say, regular categories are those categories whose subobjects "behave nicely" in the sense that subobjects of $A \times B$ (relation between $A$ and $B$) admit many of the operations we expect relations to have. For instance given relations $R$ on $A \times B$ and $S$ on $B \times C$, we can compose them to get a relation $S \circ R$ on $A \times C$.
But the star of the show is existential quantification. Given a relation $R$ on $A \times B$, there's a subobject of $B$, called "$\exists_A R$" so that, morally, $b \in \exists_A R$ iff there exists an $a$ with $(a,b) \in R$. You can make this precise with arrows instead of elements, but if you read $\in$ as a generalized element then this notation works in any regular category and is literally true.
This can be pushed even further so that, in any regular category, we can interpret theories regarding relations (for instance, the theory of posets, etc.) provided the axioms for those theories can be written in "regular logic". See Butz's excellent paper Regular Categories and Regular Logic for more.
There's also recently been a trend where people develop "graphical methods" for working in various categories. This is nice because often the pictures are easier to understand, and it's nice to be able to do honest calculations by checking that two drawings "are similar enough". See, for instance this talk from ACT 2021 by Tslil Clingman. (NB: that youtube video has three talks. The one I'm referencing is the first talk, which only takes the first half hour of the runtime)
I hope this helps ^_^