The noncompact Stiefel manifold is the set of $\mathbb{R}^{n \times p}$ matrices ($p \leq n$) that have rank $p$ (full rank). Based on my readings of http://press.princeton.edu/chapters/absil/Absil_Chap3.pdf, my guess is that the tangent space $T_x \mathcal{M}$ to this manifold is given by the following set $$ \{ Z \in \mathbb{R}^{n \times p} : \mathrm{rank}( X + Z ) = p \} $$ That is, the set of matrices that do not change the rank of $X$ when added.
Is this a correct statement?
This noncompact Stiefel manifold $\mathcal M$ is indeed an open set in $\mathbb R^{n\times p}$, so for every point $x\in \mathcal M$, $T_x\mathcal M \cong \mathbb R^{n\times p}$.
To see that $\mathcal M$ is an open set, let $I = 1\le i_1 < i_2< \cdots < i_p \le n$ be a multiindex and let $\det_I : \mathbb R^{n\times p} \to \mathbb R$ be the map which calculate the determinant with the $i_1$, $i_2 \cdots i_p$ column. Then $\det_I$ is continuous and
$$\mathcal M = \bigcup_I \{X\in \mathbb R^{n\times p} : \ \text{det}_I (X) \neq 0\}.$$