I first derived a special case of $k=1,~l=2,~n=3,~m=4$:
$$x=p^{1/5}+p^{2/5}+p^{3/5}+p^{4/5}$$
$$x^5-10 p x^3-10p(p+1)x^2-5p(p^2+p+1)x-p(p^3 + p^2 + p + 1)=0$$
More general case works (even though it gives much longer formulas for coefficients):
$$x=ap^{1/5}+bp^{2/5}+cp^{3/5}+dp^{4/5},~~~~a,b,c,d,p \in \mathbb{Q} $$
But after encountering some other cases, I checked with Mathematica (MinimalPolynomial) and found that much more general case also leads to quintic polynomials:
$$x=ap^{k/5}+bp^{l/5}+cp^{m/5}+dp^{n/5} \tag{1}$$
For any $k,l,m,n \in \mathbb{Z}$ - including negative exponents.
I can probably derive the polynomials for any particular combination of exponents, and I can't hope for a general formula.
But can we prove that the numbers defined by $(1)$ are always roots of some quintic (degree $5$) polynomial with rational coefficients?
As a random example, we have:
$$x=13^{-11/5} - 3 \cdot 13^{-13/5} + 2 \cdot 13^{3/5} - 7 \cdot 13^{19/5}=-119687.54649388942204$$
Which is a root of:
$$302875106592253 x^5+53764811821110 x^3+98398475724403213219380 x^2+ \\ +24203484201970417908238969616010 x +7441789985038797830837836893979741390939$$
We prove the following slightly more general statement:
(By moving powers of $p$ into/out of the coefficient, we can restrict the exponents $k,l,m,n$ to the set $\{0,1,2,3,4\}$.)
Consider the vector space $V$ with basis $\{1,p^{1/5},p^{2/5},p^{3/5},p^{4/5}\}$ over $\mathbb Q$. Then each element of $S = \{1,x,x^2,x^3,x^4,x^5\}$ can be interpreted as a vector in $V$. Since $V$ is a vector space of dimension $5$, set $S$ cannot be linearly independent--thus some linear combination of these vectors must equal $0$. This gives us a corresponding polynomial of degree at most $5$.