In flat space I may write a delta function as
$$\delta^{d} (x-y) = \lim_{t \to 0} (4\pi t)^{-d/2}e^{-\frac{(x-y)^2}{4t}}.$$ So, does the curved space analogue look like
$$\delta^{d} (x,y) = \lim_{t \to 0} (4\pi t)^{-d/2}e^{-\frac{\sigma(x ,y)}{4t}}$$ where $\sigma$ is the geodesic distance square between $x, y$ ?
The Dirac-delta function $\delta(x,y)$ is defined by the condition that it is localised at a point $y$, and when its product with a test function $f(x)$ is integrated it yields the value of the test function at the point $y$.
Now on a Riemannian manifold, the invariant volume is $\mathrm{d}^dx \sqrt g $ where $g$ is the determinant of the metric. The invariant delta function should thus be
$\delta^{d} (x-y) = \frac{1}{\sqrt g}\lim_{t \to 0} (4\pi t)^{-d/2}e^{-\frac{(x-y)^2}{4t}}.$
This probably doesn't answer your question fully, but could be helpful if you need it for computation.