I read that the tangent space $T_p(\mathbb{R}^n)$ at $p$ in $\mathbb{R}^n$ is the vector space of all arrows emanating in $p$.
Does this mean that we $p$ is the origin in $T_p(\mathbb{R}^n)$ and that we can identify $T_p( \mathbb{R}^n) \cong \mathbb{R}^{n-1}$?
I think this is geometrically more clear in my head but don't know how to make it formal.
Yes, in this interpretation of the tangent space, we place the origin at $p$. But the dimension doesn't decrease, so we have $T_p(\Bbb R^n)\cong \Bbb R^n$. (Consider $T_0(\Bbb R^n)$ for simplicity. There are $n$ full dimensions of vectors to choose from.)
Yes, a surface in $\Bbb R^3$ will have an $\Bbb R^2$ tangent space at each point, but that's not what this is about. This is about $\Bbb R^n$ itself being the manifold we study.