Update: Qiaochu Yuan points out in the comments that the title of the question is misleading, as homological algebra did not begin with long exact sequences as I'd thought.
(Original question follows.)
I want to understand how anyone knew that the long exact sequence in homology was something to go looking for. It seems like every time I push it back a step I just end up with more questions, though...
I know that the long exact sequence in homology comes from short exact sequences of chain complexes. The motivating example I've generally seen for this is something like this: Let $B$ be a (simplicial, CW, etc.) complex and let $A$ be a subcomplex of $A$. The inclusion induces a map of chain complexes $C_\bullet A \to C_\bullet B$, and so we can get a short exact sequence $0 \to C_\bullet A \to C_\bullet B \to C_\bullet B / C_\bullet A \to 0$. If we write $H_\bullet(B, A) := H_\bullet(C_\bullet B / C_\bullet A)$ then we get induced maps $H_n(A) \to H_n(B) \to H_n(B, A)$.
Ignore for the moment that it seems to be a pretty far stretch to get from there to the idea that a long exact sequence might exist in homology. So far as I can tell $H_n(B, a)$ is not a natural object to study from a geometric perspective, and the only purpose of defining it is that it (tautologically) gives us a short exact sequence.
But then how do we know that short exact sequences are something we want to study? The only motivation I know of for wanting to look at a short exact sequence is that it can lead to a long exact sequence in homology, but how do you discover that fact without knowing to start by looking at short exact sequences?
Of course even once we've started looking at short exact sequences, the existence of a connecting map seems far from obvious.
Can anyone give some insight as to how people ever came up with this stuff?
In my opinion, the relative singular homology group $H_n(B, A)$ is naturally a geometric object. As Stefan H. says in the comments, the idea of considering the boundary of a relative cycle in $(A, B)$ as a cycle of one dimension less in $B$ is quite natural.
But why is this connected to the short exact sequence? A quotient of chain groups (the relative group) ought to be related to a quotient of the underlying spaces.
In the relative group $C_n(B, A)$, the condition for a chain to be a cycle is relaxed from the boundary being $0$ to the boundary being a chain in the subspace $A$. Under mild assumptions on the pair $(A, B)^\dagger$, the quotient map $B \to B/A$ induces an isomorphism $$ H_n(B, A) \overset{\sim}{\longrightarrow} \tilde{H}_n(B/A). $$ When the subspace $A$ is quotiented to a point, the boundary of a chain in $B$ maps to that point in $H_n(B/A)$, or $0$ in the reduced group.
$^\dagger$ $A$ is closed and is a deformation retract of a neighborhood $A'$ in $B$