Let $\text{GL}_2(\mathbb{F}_p)$ act on $\mathbb{F}_p^2,$ the set of $2$-vectors with entries in $\mathbb{F}_p$, by matrix multiplication.
$[\,$Prove that$\,]$ for any $\mathbf{A}\in\text{GL}_2(\mathbb{F}_p)$ with $\text{ord}\,\mathbf{A}=p$, there exists $\mathbf{v}\in\mathbb{F}_p^2\setminus \{\mathbf{0}\}$ such that: $$\mathbf{A}(\mathbf{v})=\mathbf{v}\quad [\,\text{done}\,]$$
Deduce that $\mathbf{A}$ is conjugate in $\text{GL}_2(\mathbb{F}_p)$ to $\begin{pmatrix} 1\quad1\\0\quad1\end{pmatrix}$
$\big[$taken from $2012\;7\text{E}$ of part $\mathsf{IA}$ of the Cambridge tripos$\,]$
I am having a hard time seeing how one would deduce the claim from the previous statement. I can prove that $\mathbf{A}$ is conjugate to said matrix using results from linear algebra, but I think this is missing the point of the question. Would anyone care to offer insight on this? Thank you for any suggestions/help.
Note: just to stress the question: the problem is in the context of a purely group-theoretic exam; I am after a solution which uses no linear algebra at all.
This is equivalent to showing that there's a basis with respect to which the matrix of $A$ is $\begin{bmatrix} 1&1 \\ 0&1\end{bmatrix}$. If you choose the first basis vector to be $v$, then the first column of the matrix will be $\begin{bmatrix} 1 \\ 0\end{bmatrix}$.
Now, the other diagonal entry will have to be $1$, since taking powers of an upper triangular matrix just raises the diagonal entries to that power, and $1$ is the only residue that satisfies $a^p=1$. Now you have that the matrix is conjugate to $\begin{bmatrix} 1&a \\ 0&1\end{bmatrix}$. Now you just have to conjugate this to make $a=1$.