I'm reading through Milnor's Topology from the Differentiable Viewpoint, and one page $21$ he does the following
Homotopy Lemma: Let $f, g : M \to N$ be smoothly homotopic maps between manifolds of the same dimension, where $M$ is compact and without boundary. If $y \in N$ is a regular value for both $f$ and $g$, then $$\#f^{-1}(y) = \#g^{-1}(y) \ \ \ \text{(mod $2$)}$$
Proof: Let $F : M \times [0, 1] \to N$ be a smooth homotopy between $f$ and $g$. First suppose that $y$ is also a regular value for $F$. Then $F^{-1}(y)$ is a compact $1$-manifold with boundary equal to $$F^{-1}(y) \cap (M \times \{0\} \cup M \times \{1\}) = f^{-1}(y) \times \{0\} \cup g^{-1}(y) \times \{1\}$$
Thus the total number of boundary points of $F^{-1}(y)$ is equal to $$\#f^{-1}(y) + \#g^{-1}(y)$$
Now I can see that $$F^{-1}(y) \cap (M \times \{0\} \cup M \times \{1\}) = (F^{-1}(y) \cap M \times \{0\}) \cup (F^{-1}(y) \cap M \times \{1\})$$ but I can't conclude that $$ (F^{-1}(y) \cap M \times \{0\}) \cup (F^{-1}(y) \cap M \times \{1\}) = f^{-1}(y) \times \{0\} \cup g^{-1}(y) \times \{1\}$$
Also I can't see why $F^{-1}(y)$ is compact, and I can't see why we can conclude that the total number of boundary points of $F^{-1}(y)$ is equal to $\#f^{-1}(y) + \#g^{-1}(y)$
How could I fill in the missing details of this proof?
The two different right-hand sides you've exhibited above are the same thing, rewritten. For instance, the part of the preimage $F^{-1}(y)$ that intersects the bottom $M \times \{0\}$ must be $f^{-1}(y) \times \{0\}$ because the homotopy is $f$ at the bottom. The same thing goes for $g$ and the top of the cylinder.
Now, your two left-hand sides are also equal by distributing $\cap$ over $\cup$. Hence all four things are equal, as Milnor claims.
Finally, the same sort of thing gives the boundary count. Your second right-hand side gives the boundary in terms of "separated" preimages of $f$ and $g$. What matters here is that the union is a disjoint union, because the two ends of the cylinder don't intersect.
Edit: if it helps, Milnor is tacitly assuming that $F$ is $f$ at the bottom and $g$ at the top, so $F(-,0)=f$ and $F(-,1)=g$. This might help explain those set-equalities.