Most proofs of $$ \left\vert ~\mathrm{GL}(n,\mathbb{F}_p) ~\right\vert = \prod_{k=0}^{n-1} (p^n-p^k) $$ I have seen so far, are done by counting the possibilities to build up invertible matrices i.e. counting the number of bases $\mathbb{F}_p^n$ has. ( as a notable exception: here is a different proof ) I would consider this a constructive proof, since it also gives you a method to list all the matrices in $\mathrm{GL}(n,\mathbb{F}_p)$, if you wanted to do that.
Using the result on the order of $\mathrm{GL}(n,\mathbb{F}_p)$ I have seen proofs of $$ \left\vert ~\mathrm{SL}(n,\mathbb{F}_p) ~\right\vert = \frac{1}{p-1}\prod_{k=0}^{n-1} (p^n-p^k) = p^{n-1} \prod_{k=0}^{n-2} (p^n-p^k) $$ But the ones I have found so far were not constructive. So I'm looking for a constructive proof of this, similar to the one for $\mathrm{GL}$.
I am not sure what you mean by constructive, but consider the following.
List all the matrices in ${\rm GL}_n(\Bbb F_p)$ (for instance, by listing bases of $\Bbb F_p{}^n$). Declare that two matrices are equivalent if the first $n-1$ rows are equal and the $n$-th of one can be obtained from the $n$-th of the other by multiplying it by a non-zero scalar $\alpha\in\Bbb F_p^\times$.
Since any row of an invertible matrix cannot be made up entirely of $0$'s, each equivalence class consists of $p-1=|\Bbb F_p^\times|$ elements and exactly one of these has determinant $1$.
In this way you show "constructively" that $\left|{\rm SL}_n(\Bbb F_p)\right|=\frac1{p-1}\left|{\rm GL}_n(\Bbb F_p)\right|$.
Alternatively, you can proceed exactly as for the ${\rm GL}_n(\Bbb F_p)$ case, by counting bases, except that at the last step instead of taking just any vector linearly independent with the others (of which you have $p^{n}-p^{n-1}$ choices) you count only those such that the resulting matrix has determinant one. Essentially by the same argument, namely multiplying a vector by a scalar changes the determinant by that scalar, the final choices are just $\frac{p^{n}-p^{n-1}}{p-1}$ and you get your formula.