Is it okay to divide something by a random variable that can take on the value of 0 with probability greater than 0?

Question

Is it okay to divide something by a random variable that can take on the value of 0 with probability greater than 0?

For example, we know that, by using Slutsky's Theorem, If $\hat{p^*} = \frac{Y}{n}, Y \sim Bin(n,p)$,

then $\frac{\hat{p^*}-p}{\sqrt{\hat{p^*}(1-\hat{p^*})/n}} \rightarrow^d N(0,1) $

(i.e. $\frac{\hat{p^*}-p}{\sqrt{\hat{p^*}(1-\hat{p^*})/n}}$ converges in distribution to $N(0,1)$ ).

But here, $\hat{p^*}$ is a random variable that can take on the value of $0$ with probability greater than $0$, which means that the probability that $\sqrt{\hat{p^*}(1-\hat{p^*})/n}$ is equal to $0$ is also greater than $0$. so I am not sure whether it is indeed mathematically okay to write an expression like $\frac{\hat{p^*}-p}{\sqrt{\hat{p^*}(1-\hat{p^*})/n}}$ , because we are not supposed to have $0$ in denominator of a fraction.

So to summarize, is it okay to have an expression like $\frac{c}{Y}$,

where

$c=$ constant, and

$Y=$ a random variable that can take on the value of 0 with a probability greater than $0$.

I hope my question makes sense.

Thank you,

Answer 1

It stems from an asymptotic result (De-Moivre-Laplace theorem). If $X_1, X_2,...$ are i.i.d Bernoulli r.v.s with $p$, then for $n\to \infty$ you have $$ \frac{\sum_{i=1}^n X_i - np}{\sqrt{np(1-p)}} \xrightarrow{D} \mathcal{N}(0,1). $$ Namely, you are right that for any finite $n$ (even very large ones) $$ \mathbb{P}(n^{-1}\sum_{i=1}^n X_i = 0) >0. $$ However, for $n\to \infty$ and $p\in(0,1)$, $$ \lim_{n\to \infty} \mathbb{P}(n^{-1}\sum_{i=1}^n X_i = 0) =0. $$ In other words, the sample variance of $\hat{p}$ that is written as $n^{-1}\hat{p}( 1 - \hat{p})$ converges in probability (in this case, you got the almost surely converges as well due to SLLN) to $p(1-p) > 0$. And $\sqrt{n}(\bar{X}_n - p) \xrightarrow{D} \mathcal{N}(0, p(1-p))$, hence you can use Slutsky and the continuous mapping to get $$ \frac{\sqrt{n}(\bar{X}_n - p)}{\sqrt{\hat{p}(1-\hat{p}}} \xrightarrow{D} \frac{1}{\sqrt{ p(1-p) }}\mathcal{N}(0, p(1-p)) = \mathcal{N}(0,1). $$ Meaning that every use of this result for finite sample sizes is just an approximation, hence it "is not OK" in this sens to divide by such a r.v. However if you ever took a course in advanced econometrics or any other applied math or stats course, then you see tones of approximations and different rule of thumbs that "abused" the mathematical foundations for practical reasons.

Answer 2

We're usually taught the oversimplification that you "can't" divide by zero. This isn't true. Just watch me: $\frac 1 0$. In order to get an accurate statement out of this idea, we need to be more precise with what we mean by "you can't do it".

One possible meaning is that if you know $ab=ac$, then you can conclude $b=c$, unless $a=0$, since for example $0\cdot 1=0\cdot 2$. This isn't really about dividing, it's about what deductions can be made about multiplying.

In your case however, the relevant meaning is that dividing by zero is undefined. In other words, when you define $Z=X/Y$, where $Y$ can be zero with non-zero probability, you actually haven't defined $Z$. More specifically, since a random variable is formally a function on a set (the probability space), you need to define $Z(\omega)$ at each $\omega$. You haven't done that, since I have no idea what $Z(\omega)$ should be for values of $\omega$ for which $Y(\omega)=0$. It's as if you defined a function on $\mathbb N$ by saying "$f(n)$ is the position of the fourth $7$ in the decimal expansion of $n$". I have no idea how you want to define $f(372)$.

So what you need to do here is define your random variable more precisely, and then see if your argument applies to the resulting object.

---

Let's see how we can do this with your problem. I'll write $\hat{p_n}$ for your $\hat{p^*}$ to make it clear it's a sequence. You have the tentative statement

$$Z_n:=\sqrt n \frac {\hat{p_n} - p}{\sqrt{\hat{p_n}(1-\hat{p_n})}}\to \mathcal N(0,1)$$

Of course you expect this to happen because $\hat{p_n}$ has the law of the average of $n$ independent Bernoulli trials, so without the denominator you would have something that converges in law to $X\sim \mathcal N(0, p(1-p))$, and the denominator in $Z_n$ converges in law to $\sqrt{p(1-p)}$, and $X$ divided by that has the law $\mathcal N(0, 1)$, so by Slutsky's theorem you should be away.

As I said, our task is to redefine $Z_n$ more precisely (or indeed define it at all) so that this argument works. Can we do this? We could do something like say "when the denominator is zero, $Z_n=0$". The worry is that then Slutsky's theorem might not apply anymore. Well, let's look at what Slutsky's theorem says exactly:

$X_n/Y_n\to X/c$ when $X_n\to X$, $Y_n\to c$, as long as $c\neq 0$

Here $c$ is definitely non-zero (except when $p=0$ or $1$), so that's fine. Notice that the attempt at patching up the definition of $Z_n$ I mentioned in the last paragraph doesn't work, since then $Z_n$ is no longer of the form $X_n/Y_n$, it's of the form "$X_n/Y_n$ some of the time, but $0$ the rest of the time". Instead, we need to go one level deeper and redefine the denominator itself so it's never zero. Let:

$$\hat{\sigma_n}=\begin{cases} \sqrt{\hat{p_n}(1-\hat{p_n})} &\text{ when } \hat{p_n}\neq 0, 1 \\ 1 &\text{ otherwise} \end{cases}$$

Note that $1$ could be replaced by any non-zero number. Now we can define $Z_n=\frac{\sqrt n (\hat{p_n} - p)} {\hat{\sigma_n}}$ and see if Slutsky's theorem applies. The numerator still converges to $\mathcal N(0,1)$, and I'm sure you can show that $\hat{\sigma_n}$ converges in law to $\sqrt{p(1-p)}$.

Although it may seem too good to be true that we were able to get out of a mathematical quandary by arbitrarily defining away the thing that was bothering us, the point is that that thing becomes rarer and rarer as $n\to\infty$, so our ad hoc duct-tape solution becomes less and less visible and vanishes in the limit.

Answer 3

The definitions of Slutsky's Theorem I have seen, specifically prohibit denominators that take the value $0$ with probability greater an 0.

In the case you mention, one can use $p$ instead of its estimate. Then a second argument can be made that $\hat p = Y/n$ converges to $p.$ The deprecated "95%" CI of the form $\hat p \pm 1.96 \sqrt{\hat p(1-\hat p)/n},$ which is based on this shaky argument, has been shown to have very bad coverage probability (far from 95%) in many cases of practical interest. See this page for more on the CI and relevant references.

"Theoretically," you may find arguments that appear to circumvent dividing by 0 in Slutsky's theorem, but the practical consequences of the result may remain in doubt.

If you try to check questionable arguments computationally, you might not detect difficulties. Most software languages have 'protections' against overflow and underflow that simply "patch" difficulties, often without warning messages. For example, in R and many other computer languages 0/0 returns a non-number NaN, but 0^0returns 1 (roughly, on the grounds that the base 0 must be an underflow and not a "real" $0$.) Computers deal with a very large, often-useful subset of the rational numbers, not with all real numbers.