What is $\int_{\mathbb{R}^r}\delta\left(f(z)\right) g(z) dz$?
Here $\delta(\cdot)$ is a $d$-dimensional Dirac-delta function, $z$ is a $r$-dimensional variable ($r<d$). $f:\mathbb{R}^r\to\mathbb{R}^d$ is a differentiable function. $g(z)$ is a bounded. To make things simple, we assume that there exists only one $z$ such that $f(z)=0$.
I know this is going to be infinite but I want to know its divergence rate.
We can write $\delta\left(f(z)\right)$ as the limit of a Gaussian distribution: $$ \lim_{\gamma\to0}\mathcal{N}(x|f(z),\gamma I) $$ I guess $\int_{\mathbb{R}^r} \mathcal{N}(x|f(z),\gamma I) g(z)dz = \Theta(\gamma^{(d-r)/2})$, meaning they are the same order infinity. Am I correct? If so, how can I prove it?
Your question doesn't make sense.
Do you see that it is natural to define the Dirac delta by $$\int_{-\infty}^\infty \delta(x) \varphi(x)dx = \lim_{a \to 0^+} \int_{-\infty}^\infty \frac{1_{x \in (-a,a)}}{2a} \varphi(x)dx = \lim_{a \to 0^+}\frac{1}{2a} \int_{-a}^a \varphi(x)dx = \varphi(0)$$ where the last equality assumes $\varphi$ is continuous at $x=0$.
Thus for $f: \mathbb{R} \to \mathbb{R}$ it is natural to define $\delta(f(x))$ by $$\int_{-\infty}^\infty \delta(f(x)) \varphi(x)dx = \lim_{a \to 0^+} \int_{-\infty}^\infty \frac{1_{f(x) \in (-a,a)}}{2a} \varphi(x)dx$$ If $f$ is differentiable, has only one zero at $x=x_0$ and $f'(x_0) \ne 0$ and $\varphi$ is continuous at $x_0$ then $$ \lim_{a \to 0^+} \int_{-\infty}^\infty \frac{1_{f(x) \in (-a,a)}}{2a} \varphi(x)dx =\lim_{a \to 0^+} \frac{1}{2a}\int_{x_0-\frac{a}{|f'(x_0)|}}^{x_0+\frac{a}{|f'(x_0)|}} \varphi(x)dx= \frac{\varphi(x_0)}{|f'(x_0)|}$$ This generalizes in dimension $d$ with $$\int_{\mathbb{R}^d} \delta(F(x)) \Phi(x)dx = \lim_{a \to 0^+} \int_{\mathbb{R}^d} \frac{1_{\|F(x)\| < a}}{a^d S_d} \varphi(x)dx=\frac{\Phi(x_0)}{|\det(\nabla F(x_0))|}$$ whenever $\Phi : \mathbb{R}^d \to \mathbb{R}^d $ is continuous at $x_0$ and $F : \mathbb{R}^d \to \mathbb{R}^d $ has only one zero at $x_0$, is differentiable and the matrix $\nabla F(x_0))$ is inversible and $S_d = \int_{\mathbb{R}^d} 1_{\|x\| < 1}dx$