Idea behind applications of homotopy and (co)homology groups

Question

What is the idea behind applications of homotopy and (co)homology groups?

When algebraic invariants are used to solved problems of the form "are two spaces $X$ and $Y$ homeomorphic?", this is clear: if $X$ and $Y$ have different invariants, they can't be homeomorphic. So here one can clearly extract an idea. I'm talking about different applications of homotopy and (co)homology groups.

Hatcher's book, for instance, gives several applications of fundamental groups: the fundamental theorem of algebra, Brouwer's fixed point theorem in dimension 2, and the Borsuk-Ulan theorem in dimension 2. All these proofs are just random proofs by contraction, it seems, and I am not able to extract the common idea underlying these proofs. I can't spot any similarity between these proofs, except the feature that they are all proofs by contradiction.

However, May's text book says in the first section "What is algebraic topology?":

In oversimplified outline, the way homotopy theory works is roughly this.

1. One defines some algebraic construction $A$ and proves that it is suitably homotopy invariant.
2. One computes $A$ on suitable spaces and maps.
3. One takes the problem to be solved and deforms it to the point that step 2 can be used to solve it.

So this suggests there a method or common pattern underlying all the applications of homotopy and (co)homology groups. What is it that makes step 3 work, i.e., that one can reduce many problems to invariant calculations?

nLab suggests the usefulness of these invariants comes from the fact that they are functorial: if a certain statement is true about the category of topological spaces, then it is preserved by the algebraic invariant, so that it yields a true statement about the category of groups. The contrapositive is used in Wikipedia's phrasing of the proof of Brouwer's fixed point theorem.

But it seems to me the others proofs don't rely on exploiting functoriality in any way. This application of the de Rham cohomology is also a proof by contradiction and uses functoriality - but again it seems a bit random and I am not able to extract the idea of the usage of de Rham cohomology.

Let me end by a quote similar to May's, which I found in McLarty's article The Rising Sea:

This is the really deep simplification Grothendieck proposed. The way to understand a mathematical problem is to express it in the mathematical world natural to it—that is, in the topos natural to it. [...] I would outline this:

1. Find the natural world for the problem (e.g. the étale topos of an arithmetic scheme).
2. Express your problem cohomologically (state Weil’s conjectures as a Lefschetz fixed point theorem).
3. The cohomology of that world may solve your problem, like a ripe avocado bursts in your hand.

To state my question a last time: why is it that so many problems can be stated or proved "cohomologically"?

Answer 1

The question is a bit vague for my taste. I like May's description, except, I would go one step further and say that "deformations" should be understood here more broadly than just in the realm of the homotopy theory and include homology/cohomology as well. (The notion of a "deformation" would be more subtle, especially in the cohomology case.) To continue with your question:

So this suggests [that] there [is] a method or common pattern underlying all the applications of homotopy and (co)homology groups. What is it that makes step 3 work, i.e., that one can reduce many problems to invariant calculations?

Yes, for step 3 to work one needs an abundant supply of "deformations" of, say, given map (see below). In contrast, there are situations where the ability to "deform" is highly constrained and one needs to look for more complicated tools than the standard algebro-topological gadgets.

The most common deformation technique that you will see in algebraic topology is a variation on the theme of "straight-line homotopy", where given two maps $f, g: Y\to X$, provided that $X$ has a suitable notion of "line segments" between points $p, q$ parameterized by the unit interval, $s=s_{p,q}(t)$, say, $$ [p,q]=\{s(t)=(1-t)p+q: t\in [0,1]\}\subset {\mathbb R}^n, $$ one defines the straight-line homotopy by the formula $$ F(x, t)= s_{f(x),g(x)}(t), $$ e.g. $$ F(x,t)= (1-t)f(x) + t g(x). $$ Here $f$ is frequently the map that you want to analyze and $g$ is a map that you can analyze, possibly by a straightforward calculation which is much simpler to perform than the one for the map $f$ (for instance, $g$ is the constant map).

Similar "straight-line homotopy" exists between nearby maps to Riemannian manifolds and, more generally, to CW complexes. But in more pathological spaces one does not have a reasonable substitute for "line segments", say if you consider the Cantor set, Hawaiian Earrings, or Sierpinski carpet.

Of course, you still need the ingredients (1) and (2) to work. A good example (actually, one of the earliest homotopy invariants) is the notion of degree of maps which allows one to prove existence of solutions of nonlinear equations $f(x)=y_0$ for continuous maps between suitable spaces. To be more concrete, take $B$ to be the closed unit ball in ${\mathbb R}^n$ bounded by the unit sphere $S^{n-1}$ and consider continuous maps $$ f: B\to {\mathbb R}^n. $$ You would like to solve the equation $f(x)=y_0$ (thus, $n$ equations, $n$ unknowns), or, more precisely, prove that a solution exists. For maps $f$ one defines the degree $deg(f,y_0)$ (provided that $y_0\notin f(S^{n-1})$) so that it is stable under deformations of $f$ (as long as $y_0$ does not belong to the image of $S^{n-1}$ throughout the deformation) and then attempts to deform $f$ to a "better map" $g$, for which the degree is easily computable. Or, maybe you simply want to prove that the existence of a solution is "stable", i.e. does not change if you perturb the map $f$ or $y_0$ a bit. (This stability is essential if you were to solve such problems numerically.) The notion of degree works quite well for this purpose. It fails badly though if you were to use it in the case of infinite-dimensional Banach spaces (which would given you the capacity to solve some very general PDEs). In this infinite-dimensional setting you still have the line-homotopy, but the ordinary notion of degree fails because you either get zeroes or infinities if you try! One can still define a meaningful substitute for the algebro-topological degree in this case (originated by Leray and Schauder in 1930s), but it is a more complicated story.

Here is an example of such an application of degree to "stability" of solutions of the equation $f(x)=y_0$ (the example is taken from MSE):

Let $U\subset \mathbb R^n$ be an open subset and let $f:U\to \mathbb R^n$ be a function of the class $C^1$, $n>1$. Assume that the set of zeroes of the Jacobian determinant of $f$ is discrete. Prove the function $f$ must be open.

See if you can solve this problem without using any tools of algebraic topology.