If we have a negative binomial like
$N \sim NegBin(r, p)$.
Then
$(N = i \mid p) = \binom{i - 1}{r - 1} p^r (1 - p)^{i - r} \mathbb{1}(\{i \in \{r, r + 1, \dots)$
We are given that $p$ has a beta prior with distribution $a$ and $b$. I am trying to find the posterior distribution $[p∣N,r,a,b]$. Do I need to find the density of the negative binomial distribution first?
The negative binomial distribution is a discrete distribution so has a probability mass function (as you have stated - note there are other parametrisations of the negative binomial) rather than a density. It is the Beta distribution which has a density, proportional to $p^a(1-p)^b$.
If you have a single observation $x$ from your negative binomial distribution with $x \in \{r,r+1,\ldots\}$ then the posterior density for $p$ given $x$ will be (ignoring multiplicative constants) proportional to $p^a (1-p)^b p^r (1-p)^{x-r}$ when $0 <p<1$, with the first part coming from the Beta prior and the second part from the negative binomial likelihood.
Tidying this up gives a posterior density proportional to $p^{a+r}(1-p)^{b+x-r}$, which is the density of a Beta distribution with parameters $a+r$ and $b+x-r$.
If instead you had $n$ observations, the posterior distribution would be a Beta distribution with parameters $a+nr$ and $b+(\sum x_j)-nr$.