We see in certain places that $$\operatorname{div} \mathbf{F}|_p = \lim_{V \to \{p\}} \iint_{\partial V} \frac{\mathbf{F} \cdot \mathbf{\hat n}}{|V|} \: dS.$$
But what is $V$? Neighborhoods of $p$? Balls around $p$? How can you take a limit that is not as one number approaches another? What is the actual rigorous definition of this limit, and furthermore, divergence?
I am already familiar with the $\nabla \cdot F$ definition.
Let's put it this way. Suppose you have defined the divergence as a differential operator: that is, for a general function $\mathbf f : A \to \mathbb R^n$, where $A$ is a subset of $\mathbb R^n$, and for $\mathbf x_0 \in A \setminus \partial A$ at which $\mathbf f$ is differentiable, let the divergence of $\mathbf f$ at $\mathbf x_0$ be defined as the real number $$\operatorname{div} \mathbf f (\mathbf x_0) := \operatorname{tr} J\mathbf f(\mathbf x_0) = \sum_{i=1}^n \frac{\partial f_i}{\partial x_i}(\mathbf x_0),$$ where $J\mathbf f(\mathbf x_0)$ is the Jacobian matrix of $\mathbf f$ at $\mathbf x_0$ and $\operatorname{tr}$ indicates the trace operator.
Then the following theorem holds:
By $\operatorname{diam}A_k$ we mean the diameter of the set $A_k$, i.e. the greatest possible distance between two points in the closure of $A_k$: $$\operatorname{diam} A_k := \sup_{\mathbf x, \mathbf y \in A_k} |\mathbf x-\mathbf y|; $$ also by $\operatorname{vol}_n$ we mean the $n$-dimensional standard measure associated with $\mathbb R^n$. For $n = 3$ we're dealing with the usual notion volume.
By regular open set we mean that $A_k$ should have a "nice" border, i.e. that $\partial A_k$ should be a $(n-1)$-dimensional compact manifold of certain regularity (otherwise we will have problems defining the outward unit normal function $\hat{\mathbf n}_k$). More precisely, for an open set $U \subseteq \mathbb R^n$ to be regular, we require that it be bounded, and that for all $\mathbf x_0 \in \partial U$ there exist a neighborhood $I$ of $\mathbf x_0$, a neighborhood $J$ of the origin of $\mathbb R^n$, and a bijection $\mathbf r : J \to I$ of class at least $C^1$ such that
Of course, by $\mathbb R^n_+$ we mean the (open) region of $\mathbb R^n$ where $x^n > 0$ (or for that matter any other one of the coordinates in place of $x^n$); this is also called the $n$-dimensional upper half-space.
There is also a useful characterization of these regular open sets:
Of course, that famous theorem which states that contour lines of regular functions can be locally parametrized by maps of the same regularity prevents the border $\partial U$ from having any "sharp edges" or "corners" or other similar singular portions. That is why smooth balls are usually employed as the prototypical choice of $A_k$.
As an exercise: try to rewrite the statement of the theorem above by replacing the (discrete) sequence $\{A_k\}_{k\in\mathbb N}$ with the (continuous) family of open balls $\{B_{\varepsilon}(\mathbf x_0)\}_{\varepsilon > 0}$. (Spoiler: it becomes much simpler)