Here is the definition of Cauchy sequences (and null sequences and completion) from Lang's Undergraduate Algebra:
Let G be a group. Let F be the family of all subgroups of finite index. Let {$x_n$} be a sequence in G. We define this sequence to be Cauchy if given $H\in{F}$ there exists $n_0$ such that for m,n$\ge$$n_0$ we have $x_n$$x_m^{-1}$$\in{H}$. If {$x_n$}, {$y_n$} are two sequences, define their product to be the sequence {$x_ny_n$}. Furthermore, define {$x_n$} to be a null sequence if given $H\in{F}$ there exists $n_0$ such that for n$\ge$$n_0$ we have $x_n$$\in{H}$. We define the competition of G to be the factor group C/N - the factor group of all Cauchy sequences modulo the null sequences.
Now , under this definition I know how to prove that the Cauchy sequences form a group and the null sequences form a normal subgroup, and then considered the the factor group of al Cauchy sequences modulo the null sequences - the competition of G. However, I cannot understand how these definitions are motivated and how they tie in with the epsilon standard definitions of Cauchy sequences I have seen before. Could someone please explain the motivation behind this definition?
I believe the motivation comes from uniform spaces. Indeed in a uniform space $X$ with uniformity $\mathcal{U}$, a Cauchy sequence is a sequence $(x_n)$ such that for all $U\in \mathcal{U}$, there is $n_0$ such that for $n,m\geq n_0$, $(x_n, x_m)\in U$.
Elements of the uniformity represent $\epsilon$-neighbourhoods. Indeed, if $(X,d)$ is a metric space, it comes with a natural uniformity $\mathcal{U} = \{F \mid \exists \epsilon >0, U_\epsilon\subset F\}$ where $U_\epsilon = \{(x,y)\in X \mid d(x,y) < \epsilon\}$ (one easily checks that this does indeed form a uniformity). The axioms for uniformities are precisely there to represent properties of distances: $\Delta \subset U$ means that $d(x,x) = 0$, in other words $d(x,x)<\epsilon$ for all $\epsilon >0$, the $V$ such that $V\circ V\subset U$ represents the $\frac{\epsilon}{2}$-reasoning. Note that with this uniformity, the two notions of Cauchy sequences for a metric space coincide.
With this in mind, the definition for a Cauchy sequence in a uniform space makes total sense. Now, what's this got to do with groups and subgroups of finite index ? Well it turns out that a topological group $G$ comes with a natural uniformity as well ! (two actually, but I'll focus on one of them, the second one is dual). This uniformity is $\mathcal{U} = \{F \mid \exists V$ neighbourhood of $1, \overline{V} \subset F\}$, where $\overline{V} = \{(x,y) \in G^2 \mid xy^{-1} \in V\}$. One easily checks that this does indeed form a uniformity on $G$, and that it induces the same topology. One should think of neighbourhoods of $1$ as being closer and closer to it, and so if $xy^{-1}\in V$ it means $x$ and $y$ are $V$-close.
In this context, a Cauchy sequence in a topological group $G$ is a sequence such that for every neighbourhood of $1$ $V$, $x_nx_m^{-1}\in V$ for sufficiently large $n,m$. This is beginning to look a lot like your example. However in your example we don't have a topological group, but "just" a group. So we do what we can and choose subgroups of finite index instead, but the definition is purely analogous. This doesn't seem like a topology, but actually one can check that the uniformity defined the same way as for a topological group, replacing neighbourhoods of $1$ with finite index subgroups still produces a uniformity. The only hard part is showing that the intersection of finite index subgroups is still a finite index subgroups. With this uniformity, a Cauchy sequence is precisely a Cauchy sequence in the sense you describe, and that's where it comes from.