I need a clarification about a passage in the statement of Theorem $4.1$ of Milnor's Lectures on the H-cobordism theorem
Suppose that for some choice of gradient-like vector field $\xi$, the compact set $K_p$ of points on trajectories going to or from $p$ is disjoint from the compact set $K_{p'}$ of points on trajectories going to or from $p'$.
I don't understand why the set $K_p$ or $K_{p'}$ should be closed (hence compact since we are working in a compact manifold).
This must use the fact that we have only $2$ critical point in our manifold with boundary, $p_1,p_2$, otherwise the statement is not true (a flow line might converge to a broken flow-line).
Here is my attempt. Let $\gamma \colon \Bbb R \times W \to W$ be the flow of $\xi$. Let $\{x_n\}_n \subset K_p$ and assume that $\gamma_t(x_n)\to p$ for $t\to \infty$. Let $x_n \to x$ for $n\to \infty$. We want to prove that $\gamma_t(x)\to p$ for $t\to \infty$. We have the following estimate:
\begin{align} \mid \gamma_t(x)-p\mid &\leq \mid \gamma_t(x)-\gamma_t(x_n)\mid +\mid \gamma_t(x_n)-p\mid\\ &\leq \mid \gamma_t(x)-\gamma_t(x_n)\mid +\epsilon \end{align}
for $t>>M$.
I don't see how to use the critical assumption that we have only two critical points because I'd be tempted to use continuity of $\gamma_t$ to claim that I can bound $\mid \gamma_t(x)-\gamma_t(x_n)\mid$ by taking $x_n$ sufficiently closed to $x$.
Can someone clarify what's wrong with my reasoning?