$\renewcommand{\S}{\mathcal{S}}$ $\newcommand{\M}{{\mathcal{M}}}$ $\newcommand{\TM}{{T\mathcal{M}}}$ $\newcommand{\TS}{{T\mathcal{S}}}$ $\newcommand{\NS}{{\mathcal{NS}}}$ $\newcommand{\N}{\mathcal{N}}$ $\newcommand{\g}{\mathcal{g}}$ $\newcommand{\Volg}{\text{Vol}_\g}$ $\newcommand{\Vol}{\text{Vol}}$ $\newcommand{\VolgS}{\text{Vol}_{\g|_\S}}$
Let $(\M,\g)$ be a smooth $d$-dimensional Riemannian manifold, and let $\S\subset\M$ be a smooth compact $k$-dimensional oriented submanifold. Let $\NS$ be the normal bundle of $\S$ in $\M$.
For a sufficiently small $h>0$, define $\S_h := \{ \exp_p(v) : p\in \S, v\in \NS, |v|\le h \}.$
Let $f:\M \to \mathbb{R}$ be a continuous function. I am trying to prove the following:
$$ \lim_{h \to 0} \frac{\int_{\S_h} f}{\Volg(\S_h)}=\frac{\int_{\S} f}{\VolgS(\S)},$$
where the integrals are w.r.t the Riemannian volume forms on $\S_h$,$S$ defined by $\g,\g|_{\S}$ respectively. (Recall $\S$ is oriented).
Edit:
Let's start with the case of a $k$-cube embedded in $\mathbb{R}^d$ in the standard way. The general case should follow by an approximation argument, since "locally, everything is Euclidean".
Here is a proof for the Euclidean case:
Let's use Fubini theorem: Suppose $\S \subseteq \mathbb{R}^k \times \{\bar 0^{d-k}\} \subseteq \mathbb{R}^d$, where $S$ is a product of $k$ intervals. Then $\S_h=S \times [-h,h]^{d-k}$.
Let $\epsilon >0$, and let $(x,y)\in S\times [-h,h]^{d-k}$.
$$\frac{\int_{\S_h} f}{\Vol(\S_h)}= \frac{\int_{[-h,-h]^{d-k}}\big(\int_{S} f(x,y) dx\big)dy }{(2h)^{d-k}\Vol(S)} =\frac{\int_{[-h,-h]^{d-k}} g(y) dy }{\Vol([-h,-h]^{d-k})} ,$$
where $g:[-h,-h]^{d-k} \to \mathbb{R}$ is defined by $$ g(y)=\frac{\int_{x \in S} f(x,y)dx}{\Vol(S)}.$$
Since the domain is compact, $f$ is uniformly continuous, so $g$ is continuous. This implies
$$ \lim_{h \to 0} \frac{\int_{[-h,-h]^{d-k}} g(y) dy }{\Vol([-h,-h]^{d-k})} =g(\bar 0)=\frac{\int_{x \in S} f(x,0)dx}{\Vol(S)}=\frac{\int_{\S} f}{\Volg(\S)}.$$
We will consider the following case :
(1) If $p:=(x,h)\in [0,l]\times [-1,1]$, then prove that
$$ \lim_r\ \frac{\int_{[0,l]}\int_{[-r,r]} f dhdx }{2rl} =\frac{T}{l},\ T:= \int_{[0,l]} f(x,0)\ dx $$
Proof :
Since $f$ is uniform continuous, then for $\varepsilon$ there is $\delta$ s.t. $d(p,q)<\delta$ implies $|f(p)-f(q)|<\varepsilon$.
Hence \begin{align*} \frac{\int_{[0,l]}\int_{[-r,r]} f(x,0)-\varepsilon\ dhdx }{2r} &<\frac{\int_{[0,l]}\int_{[-r,r]} f dhdx }{2r} \\&<\frac{\int_{[0,l]}\int_{[-r,r]} f(x,0)+\varepsilon\ dhdx }{2r} \end{align*}
so that $$T-\varepsilon l <\frac{\int_{[0,l]}\int_{[-r,r]} f dhdx }{2r} < T+\varepsilon l $$
(2) Assume that there is a cover $\{E_i\}$ for $S$ s.t. (i) $E_i\subset S,\ 1\leq i\leq N$ is homeomorphic to closed ball, (ii) $E_i\cap E_j$ has measure $0$, and (iii) $E_i$ have same volumes.
(3) If $V_i(r):=\{\exp_pv|p\in E_i,\ v\perp T_pS,\ |v|\leq r\}$, then $$ \lim_r\ \frac{\int_{V_i(r)} f}{{\rm vol}\ V_i(r)} =\frac{\int_{E_i} \ T}{{\rm vol}\ E_1} $$
Note that $$ t_i(r){\rm vol}\ V_i(r) :={\rm vol}\ V_1(r) $$ and $$ \lim_r\ t_i(r)=1$$
(4) So \begin{align*}\frac{\sum_i \ \int_{V_i(r)} f }{\sum_i V_i(r)} &=\frac{\sum_i \ \int_{V_i(r)} f }{[\sum_i \ 1/t_i(r) ]V_1(r)}\\& = \frac{1}{\sum_i \ 1/t_i(r)} \sum_i \ \frac{ \int_{V_i(r)} f }{ t_iV_i(r) } \\&\rightarrow \frac{1}{N} \sum_i \ \frac{\int_{E_i} f}{{\rm vol}\ E_i } \end{align*}
(5) More explanation on (3) : A Riemannian metric $g$ on $X:=V_i(r)$ is close to $g'=g|S + g_0$ where $g_0$ is a canonical metric on $\mathbb{R}^{n-k}$ where ${\rm dim}\ S=k$.
Given $\varepsilon$ there is $r$ s.t. $$ (1-\varepsilon )d_{g'} \leq d_g\leq (1+\varepsilon ) d_{g'}$$
so that $$ (1-\varepsilon)^n \leq \frac{ H^n(X,d_g) }{H^n(X,d_{g'} )} \leq (1+\varepsilon )^n $$ where $H^n(X,d_g)$ is a $n$-dimensional Hausdorff measure wrt a distance $d_g$
So
$$ \frac{\int_{(X,d_{g'} )} f +O(\varepsilon ) \int_{ (X,d_{g'} )} f }{ {\rm vol}\ (X,d_{g'}) +O(\varepsilon ){\rm vol}\ (X,d_{g'}) } = \frac{ \int_{(X,d_g)} f }{{\rm vol}\ (X,d_g)} $$