I am working with a $4$-dimensional representation of $D_5$.
The $4D$ representations of the generators of $D_5$, $r$ (a rotation by $\pi/5$) and $p$ (the reflection $x \rightarrow x, y \rightarrow y$) as: $$ \mathcal{R}(r) = \left (\begin{array}{ccc} 0 & 0 & 0 & -1 \\ 1 & 0 & 0 & -1 \\ 0 & 1 & 0 & -1 \\ 0 & 0 & 1 & -1 \\ \end{array} \right ), $$
$$ \mathcal{R}(p) = \left (\begin{array}{ccc} 0 & 0 & 0 & 1 \\ 0 & 0 & 1 & 0 \\ 0 & 1 & 0 & 0 \\ 1 & 0 & 0 & 0 \\ \end{array} \right ). $$
I can kinda derive these by inspection/brute force but if anyone knows of an elegant derivation, I'm all ears.
If I take $4$ linearly indepedent vectors in $4D$: $$e_1 = \left (\begin{array} & 1 \\ 0 \\ 0 \\ 0 \\ \end{array} \right ), \, e_2 = \left (\begin{array} & 0 \\ 1 \\ 0 \\ 0 \\ \end{array} \right ), \, e_3 = \left (\begin{array} & 0 \\ 0 \\ 1 \\ 0 \\ \end{array} \right ), \, e_4 = \left (\begin{array} & 0 \\ 0 \\ 0 \\ 1 \\ \end{array} \right ),$$
then I get $\mathcal{R}(r)\,e_1 = e_2$, $\mathcal{R}(r)\,e_2 = e_3$, $\mathcal{R}(r)\,e_3 = e_4$,
BUT
$\mathcal{R}(r)\,e_4 = \left ( \begin{array} & -1 \\ -1 \\ -1 \\ -1 \\ \end{array} \right ). $
What is this? Why don't I get $e_1$ back (edit: $e_1$ comes back after another application of $r$, which makes sense because we need $5$ fold rotation since we are in $D_5$)?
But what does this vector mean, physically, in terms of the basis that I gave? Like, if I represent every vertex of the pentagon with these basis vectors?
I'm not sure that the following is what is giving you a headache, but this won't fit into a comment so an answer it is.
One of your questions seems to be what has happened to the vertices of the original pentagon when we moved to this 4D representation. The short answer is that they sorta dropped out of sight, but that is not the whole truth either. Those vertices have served in so many roles here that it is not clear which role you are interested in.
The first role the pentagon played was simply to define the group $D_5$. But, when you started doing representation theory the focus sorta shifted to various ways of writing that abstract group as groups of matrices.
This 4D representation $V$ is, as a complex (or real!) representation, the direct sum, $V=W_1\oplus W_2$, of two non-isomorphic irreducible 2D representations of $D_5$. In both those representations the element $p$ acts as a reflection. But in $W_1$ the element $r$ acts as a rotation by $72$ degrees, whereas in $W_2$ it acts as a rotation by $144$ degrees.
I gathered that the direct sum $V=W_1\oplus W_2$ became a focus in the paper you cited because in $V$ we can write the matrices using integers only. If we only looked at the irreducible components $W_1$ and $W_2$ then matrix entries from the ring $\Bbb{Z}[\phi], \phi=(1+\sqrt5)/2,$ would be forced upon us. If you want to think that those vertices can only live in a rep where $r$ acts as a rotation by $72$ degrees, then look for them inside that subspace $W_1$. It is the direct sum of the eigenspaces of $\mathcal{R}(r)$ belonging to the eigenvalues $\lambda_{1,2}=e^{\pm i2\pi/5}.$
But, those five vertices also played another role in the construction of $V$. Namely, the group $D_5$ acted on the set of those five vertices. When we turned that group action into a representation those five vertices were turned into basis vectors $f_1,f_2,f_3,f_4,f_5$ of a 5D representation $U$. As explained to you in the answers and comments to your earlier question, the 4D representation $V$ is the zero-sum subspace of $U$. The orthogonal projection $P:U\to V$ is given by the recipe $$ P(x_1,x_2,x_3,x_4,x_5)=(x_1,x_2,x_3,x_4,x_5)-\frac1{x_1+x_2+x_3+x_4+x_5}5(1,1,1,1,1). $$ So you could also argue that the vertices of that pentagon can be identified as their projections $P(f_1)=P(1,0,0,0,0)$, $P(f_2)=P(0,1,0,0,0)$ et cetera. But, somewhere along the way somebody picked a basis $e_1,e_2,e_3,e_4$ for the subspace $V$, and wrote the matrices using that basis.
Unless I made a silly mistake (getting late here), whoever did that selected $e_i=P(f_i)$ for $i=1,2,3,4$. Because $P(f_1+f_2+f_3+f_4+f_5)=P(1,1,1,1,1)=0$ we can then conclude that $P(f_5)=-e_1-e_2-e_3-e_4$.
So there are at least two possible ways of identifying those vertices as vectors of $V$. Observe that as the projection $P$ is not angle and length preserving switching to basis $\{e_1,e_2,e_3,e_4\}$ ruins geometric concepts like angles and lengths.