When determining a relation's properties, you can show counterexamples to prove that it doesn't have such properties.
But I'm interested in proving, without counterexamples, that a relation lacks a certain property. This is because sometimes I simply can't think of a counterexample (usually with antisymmetry and transitivity), so I believe that the relation does have a property and I attempt to prove it, usually leading me to dead ends and wasted time.
I realize that most relations will require different approaches. But I think this one is a good start point:
Over $\mathbb{N} \times \mathbb{N}$, $(a,b)R(c,d) \iff a -c = b -d$
I previously made a question regarding this relation. This answer proves, with a counterexample, that the relation is not antisymmetric. The counteerexample used was:
Note that we have $\;(5, 4)\, R\, (3, 2)\;\land \;(3, 2) \,R \,(5, 4),\;$ but $\,(5, 4) \neq (3,2).\;$
How could I have proven, without counterexamples, that this relation is antisymmetric? The same would be great for total.
Sometimes you can prove that a relation has another property that is incompatible with the one of interest, but in general you need a counterexample. I more useful question would be how one should go about finding a counterexample. In general there are two approaches, and they can be used together.
The first is to try to prove that the relation does have the property and see what goes wrong. I’ll use your relation $R$ as an example. Suppose that $\langle a,b\rangle\mathbin{R}\langle c,d\rangle$ and $\langle c,d\rangle\mathbin{R}\langle a,b\rangle$; we’d like to show that $R$ is antisymmetric, so we’d like to be able to conclude that $\langle a,b\rangle=\langle c,d\rangle$, i.e., that $a=c$ and $b=d$. From the hypothesis we know that $a-c=b-d$ and $c-a=d-b$. But $c-a=-(a-c)$, and $d-b=-(b-d)$, so the second equation is equivalent to the first, and all we really know about $a,b,c$, and $d$ is that $a-c=b-d$. Is that enough to show that $a=c$ and $b=d$? If it were, we could use it to show that $a-c=0$ and $b-d=0$, but that’s obviously not possible: we can certainly have $a-c=b-d$ without having them both equal to $0$. And seeing where the attempted proof fails makes it easy to cook up a counterexample: just choose natural numbers $a,b,c$, and $d$ so that $a-c=b-d\ne 0$. The failed attempt to prove that $R$ is antisymmetric is essentially a proof that if $a-c=b-d\ne 0$, the pairs $\langle a,b\rangle$ and $\langle c,d\rangle$ will be a counterexample. Now you can pick a specific counterexample, say $\langle 1,1\rangle$ and $\langle 2,2\rangle$, and answer the question very briefly: $R$ is not antisymmetric, because $\langle 1,1\rangle\mathbin{R}\langle 2,2\rangle$ and $\langle 2,2\rangle\mathbin{R}\langle 1,1\rangle$, but $\langle 1,1\rangle\ne\langle 2,2\rangle$.
The second approach is useful if you suspect that your relation doesn’t have the property: assume that $\langle a,b\rangle$ and $\langle c,d\rangle$ form a counterexample, and see what conditions $a,b,c$, and $d$ would have to satisfy. Here that means assuming that $\langle a,b\rangle\mathbin{R}\langle c,d\rangle$ and $\langle c,d\rangle\mathbin{R}\langle a,b\rangle$, but $\langle a,b\rangle\ne\langle c,d\rangle$. That in turn means assuming that $a-c=b-d$, $c-a=d-b$, and either $a\ne c$ or $b\ne d$ (or both). As before, we quickly realize that $c-a=d-b$ adds no information to $a-c=b-d$, so we’re left assuming that $a-c=b-d$, and at least one of $a-c$ and $b-d$ is not $0$. So we try choosing $a,b,c$, and $d$ so that $a-c=b-d=1$, say, perhaps by setting $a=b=2$ and $c=d=1$, and we quickly discover that we do indeed have a counterexample.
It isn’t always nearly this easy, of course, but these are the two basic ways in which I approach such problems, not just in studying relations, but in any area.