I have an expression of the form
$$ \lim_{\epsilon \to 0} \lim_{\delta \to 0} \sum_{j,k=1}^\infty f(j,k,\epsilon, \delta)$$
such that the infinite double sum is absolutely convergent, even for $\epsilon = \delta = 0$, and that $f$ is continuous with respect to both $\epsilon$ and $\delta$ for every $j,k$. Can I then swap the limits with the sum?
Of course, we might also write it as $$\lim_{\epsilon \to 0} \lim_{\delta \to 0} \lim_{n \to \infty} \lim_{m \to \infty} g(m,n,\epsilon, \delta),$$
where $g(m,n,\epsilon, \delta) := \sum_{j=1}^n \sum_{k=1}^n f(j,k,\epsilon, \delta)$, so that the question becomes whether we can bring the outer limits inside.
I think that this should be possible since we have very generous assumptions, but I am not sure about what theorem exactly implies that it is feasible, and whether the key fact is continuity of $f$ or absolute convergence of the sum or both.
One general approch is to interpret summation as integration over $(\mu,\mathbb{N^2}, \mathcal{P}(\mathbb{N^2}))$, where $\mu$ is the counting measure. So the equation in question can be written as $$\int_{\mathbb{N}^2} f(j,k,\varepsilon,\delta) \, \mathrm{d}\mu(j,k).$$ Thus, we now may apply the dominated convergence thoerem.
Your assumpations are insufficient for swapping the limits! For example define $f_\varepsilon(n) = \varepsilon n^{-1-\varepsilon}.$ This function is continuous for fixed $n \in \mathbb{N}$. Moreover, we have $\sum_{n=1}^\infty f_0(n) =0$, but $$\sum_{n=1}^\infty f_\epsilon(n) \gg \int_1^\infty f_\varepsilon(x) \, \mathrm{d}x =1.$$