Suppose that I have a function $f: D \subseteq \mathbb{R}^n \to \mathbb{R}$, $D$ is closed, $f$ assumed to be continuously differentiable.
Does $\nabla f(p)$ exist for points $p$ on the boundary of $D$?
I think the answer is negative. But I do not know the precise argument because this is so seldom talked about. Recall that, $$ {\displaystyle \nabla f(p)={\begin{bmatrix}{\frac {\partial f}{\partial x_{1}}}(p)\\\vdots \\{\frac {\partial f}{\partial x_{n}}}(p)\end{bmatrix}}.}$$
Each partial derivative is,$$ {\displaystyle {\frac {\partial f}{\partial x_{i}}}(p_{1},\ldots ,p_{n})=\lim _{h\to 0}{\frac {f(p_{1},\ldots ,p_{i}+h,\ldots ,p_{n})-f(p_{1},\ldots ,p_{i},\dots ,p_{n})}{h}}.}$$ where $p = (p_1, \ldots, p_n)$.
The problem with $p$ on the boundary is that $p$ with $p_i + h$ is out side of $D$, where $f$ is not defined. Hence the above limit is not defined.
Is this the correct way of thinking about this?
You're right about the problem that $p_i + h$ is out of boundary. It shouldn't be. Actually by definition of a differentiable function $f$ in a point $x$, the function should be defined on an open set $U$ and the $x$ should be in $U$ (https://en.wikipedia.org/wiki/Differentiable_function) and then there are some other conditions. So you can't even say your function is differentiable on the entire $D$ if it cannot be defined outside the $D$. More correct would be to say it's continuous on $D$ and differentiable on $D \setminus \delta D$ (obiviously if $D \neq \delta D$). But sometimes people for simplicity do call a function differentiable on entire $D$ (especially in mathematical physics) meaning $\forall x_0 \in \delta D \,\,\, \exists \lim\limits_{x \rightarrow x_0} \nabla f(x)$. The function $\nabla f(x)$ is defined on $D \setminus \delta D$.