Several authors (such as Jackson in his book "Classical Electrodynamics") state the following conditions at an interface between two different media:
$(\vec{D_2} - \vec{D_1})\cdot \vec{n} = \sigma$
$(\vec{B_2} - \vec{B_1})\cdot \vec{n} = 0$
$\vec{n} \times (\vec{E_2} - \vec{E_1}) = 0$
$\vec{n} \times (\vec{H_2} - \vec{H_1}) = \vec{K}$.
Now, all these seem to stem from the same principle. For instance, for the second equation, demonstrations usually go like this:
Let $V$ be a finite volume in space, $S$ the closed surface (or surfaces) bounding it and $\vec{n}$ a unit normal pointing outwards from the closed volume. Consider a "pillbox" Gaussian surface. Applying the divergence theorem on it, one has $$\iiint _V \nabla \cdot \vec{B}\,dV = \iint_{S_{top}}\vec{B} \, \cdot \vec{n}\,dS + \iint_{S_{bot}}\vec{B} \, \cdot \vec{n}\,dS + \iint_{S_{lateral}}\vec{B} \, \cdot \vec{n}\,dS,$$ in which $S_{top}$, $S_{bot}$ and $S_{lateral}$ denote the top, bottom and lateral surfaces of the cylinder. Taking the limit as the height of the cylinder goes to $0$, $$\iint_{S_{lateral}}\vec{B} \, \cdot \vec{n}\,dS = 0.$$ For the remaining terms, considering an area small enough for $\vec{B}$ to be constant, and using $\nabla \cdot \vec{B} = 0$ yields $$(\vec{B_2} - \vec{B_1})\cdot \vec{n} \, \Delta A = 0$$ or simplifying $$(\vec{B_2} - \vec{B_1})\cdot \vec{n} = 0.$$
In all this, there's something that doesn't bode well with me. I get why we make the height go to $0$, but I'm not sure how mathematically correct is the step of considering a small area (which is making it go to $0$, from what I understand) and taking $\vec{B}$ outside of the integral. At the very least, if $\Delta A \rightarrow 0$ then we should get $0 = 0$.
I've decided to post this question here, and not on a physics site, because as I mentioned before, most physics books tend to stick with the same explanation. Meanwhile, I was looking to know how would a mathematician tackle this question.