For example, if I have a function $f \colon \mathbb{R}^4 \to \mathbb{R}^2$ such that $f(x,y,z,w)=(xyz+w^2,x-w-zy^2)$, what does it mean to find the following matrix:
$\partial_{p} f(p,q)$ where $p=(x,y)$ and $q=(z,w)$
My lecturer used this notation in his notes, but never really explained what it means or gave an example. Is it just supposed to be the Jacobian of $f$ but with the columns for $z$ and $w$ removed?
Yes indeed this is just the notation for a 'part' of the Jacobian.
In general, if you have a function $f:\mathbb{R}^{d}\to\mathbb{R}^{k}$, then you define $\partial_{p}f$ with $p=(x_{1},\dots,x_{l})$ for some $l\in\{1,\dots,d\}$ as
$$\mathcal{J}_{f}(x)=:(\partial_{p}f(x),\partial_{q}f(x))$$
where $\mathcal{J}_{f}(x)$ denotes the Jacobian and $q=(x_{l+1},\dots,x_{d})$.
So in this general example $\partial_{p}f$ would be
$$\partial_{p}f(x):=\begin{pmatrix}\partial_{x_{1}}f_{1}(x) &\dots&\partial_{x_{l}}f_{1}(x)\\ \vdots&\ddots&\vdots \\\partial_{x_{1}}f_{k}(x) &\dots&\partial_{x_{l}}f_{k}(x)\end{pmatrix}$$