When you look at groups of order $p^3$ (for odd $p$) there are $2$ nonabelian ones. One is the Heisenberg group which can be seen as a semidirect product of $C_p \times C_p$ and $C_p$.
Based on some computations with GAP I see that the other one is a semidirect product of $C_{p^2}$ with $C_p$.
Can this other group be seen as a familiar matrix group?
gap> c := AllSmallGroups( 3^3, IsAbelian, false );
[ <pc group of size 27 with 3 generators>, <pc group of size 27 with 3 generators> ]
gap> c[1];
<pc group of size 27 with 3 generators>
gap> StructureDescription(c[1]);
"(C3 x C3) : C3"
gap> StructureDescription(c[2]);
"C9 : C3"
gap> c := AllSmallGroups( 5^3, IsAbelian, false );
[ <pc group of size 125 with 3 generators>, <pc group of size 125 with 3 generators> ]
gap> StructureDescription(c[1]);
"(C5 x C5) : C5"
gap> StructureDescription(c[2]);
"C25 : C5"
In a word, 'no'. Notice that $\mathrm{GL}_n(q)$ for $q$ a power of $p$ cannot have any elements of order $p^2$ unless $n>p$. Thus as $p$ grows the size of the matrix group has to grow.
It's a similar story over fields of characteristic not $p$. Any $1$-dimensional representations of the group have the centre in the kernel. The only faithful representations have degree at least $p$.
So this group has no faithful representation of degree less than $p$ over any field.
Edit: There is no matrix representation over any field, but there is is over a ring. This group is given by $$ \left\{\left.\begin{pmatrix}a&b\\0&1\end{pmatrix}\,\right|\, a,b\in \mathbb{Z}/p^2\mathbb{Z},\;a\equiv 1\bmod p\right\}.$$
I found this out looking at notes of Keith Conrad just now.