The question states:
Each time a hurricane arrives, a new home has a 0.4 probability of experiencing damage. The occurrences of damage in different hurricanes are mutually independent.
Calculate the mode of the number of hurricanes it takes for the home to experience damage from two hurricanes.
The answer is supposed to be 3, because that is when the pmf is maximized, but that is not the answer I am getting.
I am assuming that $X$ has a negative binomial distribution with $p=0.4$ and $r=2$. We want to find the number of hurricanes(trials/failures) it takes for the home to experience damage from 2 hurricanes(successes).
First, I tried:
$P(X=2)$=${3}\choose{2}$$(0.4)^2$$(0.6)^2$ = $.173$
Then, $P(X=3)$=${4}\choose{3}$$(0.4)^2$$(0.6)^3$ = $.138$
where $P(X=x)$=${r+x-1}\choose{x}$$p^r$$(1-p)^x$
However, if this is the correct formula, then 3 cannot be the answer since $P(X=3)<P(X=2)$.
I am just confused about knowing how to use this formula correctly since what I've done is obviously wrong. Any help is appreciatied!
Let's work from first principles. Let $X$ be the random number of hurricanes experienced until exactly $2$ of these result in damage. Note that $X$ must also count the $2$ hurricanes that result in damage. So we compute $$\Pr[X = 2] = (0.4)^2 = 0.16,$$ because we are looking for the first two events to result in damage, and each event is independent and has probability $0.4$ of doing so.
We can also reason that $$\Pr[X = 3] = 2(0.6)(0.4)^2 = 0.192,$$ because in order for $X = 3$ exactly, the third hurricane must be the second instance of damage; and among the first two hurricanes, one causes damage and the other; this can be done in $2$ ways (either the first causes damage and the second doesn't, or vice versa).
The above reasoning can be extended further--note that counting hurricanes stops once we observe the second event that causes damage, so if $X = 4$, for example, the fourth hurricane was the second instance of damage, and among the first three, the first hurricane to cause damage could be any one of these in exactly $3$ ways, so $$\Pr[X = 4] = 3(0.6)^2(0.4)^2 = 0.1728.$$ And in general, we must have $$\Pr[X = k] = \binom{k-1}{1} (0.6)^{k-2} (0.4)^2 = \frac{4}{9} (k-1)(0.6)^k, \quad k = 2, 3, \ldots.$$ The factor $(0.4)^2$ comes from the fact that exactly two hurricanes causing damage are observed among $k$ hurricanes. One of these is always the last hurricane observed. The other may be freely chosen among the previous $k-1$ hurricanes in $k-1$ ways, thus the factor $\binom{k-1}{1}$; finally, there are $k-2$ hurricanes that do not cause damage, each independent with probability $1 - 0.4 = 0.6$. We have in essence derived the PMF of a negative binomial distribution for this specific case of parameters.
Now, to maximize $\Pr[X = k]$, we simply take the ratio of successive probabilities: $$\frac{\Pr[X = k+1]}{\Pr[X = k]} = \frac{(0.6)k}{k-1}.$$ Then as long as this ratio exceeds $1$, the probability $\Pr[X = k+1]$ exceeds $\Pr[X = k]$; but if it is less than $1$, the next probability is decreasing. So if we solve $$\frac{(0.6)k}{k-1} = 1,$$ we get $k = 2.5$, meaning that $\Pr[X = 2] < \Pr[X = 3]$, but $\Pr[X = 3] > \Pr[X = 4]$, as we calculated above; moreover, $$\Pr[X = 4] > \Pr[X = 5] > \Pr[X = 6] > \ldots.$$ because the ratio is less than $1$ for all $k > 2.5$.
So, where did you go wrong? In a nutshell, you used the wrong parametrization of the negative binomial distribution. Namely, you used a parametrization that counts the number of hurricanes not causing damage, rather than one that counts the total number of hurricanes--i.e., your random variable is less by $2$. You can see this by observing that your calculation of $\Pr[X = 2]$ equals my calculation of $\Pr[X = 4]$--both are $0.1728$. This happened because you applied a memorized formula without carefully considering how it models the problem at hand. Instead, you must be attentive and demonstrate an understanding of how the mathematics represents what you are trying to answer. As you can see in the above explanation, we obtain a correct result by taking the time to establish clear notation, define the random variable of interest, and calculate some simple cases to ensure we build and apply the correct probability model.