Popular mistakes in probability

Question

Question: What not-trivial mistakes do students often make when solving problems in probability theory, mathematical statistics and random processes?

Some examples of wrong solutions:

Problem 1: Find distribution of $F_{\xi}(\xi)$ for continious $F_{\xi}$.

"Solution": $F_{\xi}(\xi) = P(\xi \le \xi) = 1$

Problem 2: Is it possible to guess if one of a pair of random numbers is larger with probability > $\frac{1}2$?

"Solution". Obviously no (sometimes with some blurry reasoning, mentioning symmetry). (if smb.is interested, see discussion, e.g., in https://mathoverflow.net/questions/9037/how-is-it-that-you-can-guess-if-one-of-a-pair-of-random-numbers-is-larger-with )

Problem 3: Find $Var(\min(X,Y))$ for independent $X,Y \sim exp(1)$.

"Solution". If $X \le Y$ then $\min(X,Y) = X$ and $Var(\min(X,Y)) = DX = 1$, if $X > Y$ then $\min(X,Y) = Y$ and $Var(\min(X,Y)) = DY = 1$, so in any case we got $Var(\min(X,Y)) = 1$. A more absurd version is problem 3b: find $D\xi$ for $\xi \sim Bern(p)$. "Solution": $\xi$ takes values $0$ and $1$, if it's equal to $0$, then $Var\xi = Var(0)=0$, in other case $Var\xi = Var(1)=0$, hence $Var(\xi)=0$.

Problem 4: Suppose that systems of sets $\mathcal{A}_1$ and $\mathcal{A}_2$ are independent. Are $\sigma(\mathcal{A}_1)$ and $\sigma(\mathcal{A}_2)$ independent? "Solution". Obviously yes (sometimes with some blurry reasoning).

Problem 5: $2n$ teams were divided to $2$ subgroups with n teams in each group. What is the chance that the $2$ strongest teams will play in the same subgroup?

"Solution": $\frac12$ by symmetry. (see discussion, e.g. math.stackexchange.com/questions/3369903/probability-problem-whats-the-chance-that-two-strongest-teams-will-play-in-the/3369914 )

Problem 6: "A patient goes to see a doctor. The doctor performs a test with 99 percent reliability--that is, 99 percent of people who are sick test positive and 99 percent of the healthy people test negative. The doctor knows that only 0.01 percent of the people in the country are sick. If the patient test is positive, what are the chances the patient is sick?"

"Solution": 99 percent as follows immediately from the task.

Some references: There are different paradoxes, such as Monty Hall problem, see, e.g. "Paradoxes in Probability Theory and Mathematical Statistics" by G. J. Székely. There are some interesting examples of popular mistakes in "The evolution with age of probabilistic, intuitively based misconceptions" by E. Fischbein and D. Schnarch (and references therein).

Of course, there are a huge number of mistakes that, in principle, can be made, but I mean, firstly, popular ones, and secondly, it is desirable that these were errors not on the most simple topics of combinatorics and not connected with typos and so on. What not-trivial mistakes did you see lot's of times?

Addition: uninteresting examples :

1. A coin is tossed twice at random. What is the probability of getting the same face? "Solution:" Three possible outcomes are HH, HT=TH, TT, where H = head, T = tail. So the probability is $\frac13$.

2. $E\xi^2 = (E \xi)^2$ because it's "obvious" because we are speaking about mean, and even reference: there was an equality $E \xi \eta = E\xi E \eta$ in one of the properties of expectation.

3. For independent $\xi$ and $\eta$ we have $Var(\xi + \eta) = Var(\xi) + Var(\eta)$ "hence" $Var(\xi - \eta) = Var(\xi) - Var(\eta)$.

4. density of $U[0,1]$ is $F' = I_{[0,1]}(x)$ a.e., so density of $Pois(\lambda)$ is $F'= 0$ a.e.

Answer 1

I think that a common mistake is this one which occurs when trying to apply a conditional version of the law of total expectation. If $E, A$ and $B$ are events, then $$ P(A\mid B)\neq P(A\mid B, E)P(E)+P(A\mid B, E^c)(1-P(E))\tag{!} $$ The correct version has conditonal probabilities on the events $E$ and $E^c$: $$ P(A\mid B)=P(A\mid B,E)P(E\mid B)+P(A\mid B, E^c)P(E^c\mid B)\tag{$\checkmark$} $$ This mistake occurs in...

The two-envelope paradox.

Let $X$ be any positive random variable so that $E[X]<\infty$. Let $\xi\sim \operatorname{Unif}\{0,1\}$ be a zero-one coin flip independent of $X$, and let $$ Y_1=X\cdot \xi+2X\cdot(1-\xi),\qquad Y_2=X\cdot (1-\xi)+2X\cdot \xi $$ The interpretation is that $Y_1$ and $Y_2$ are random amounts of money in two envelopes, constrained so that either $Y_1=2Y_2$ or $Y_1=\frac12 Y_2$ with equal probability.

Imagine opening envelope $Y_1$, and then deciding whether or not to switch to envelope $Y_2$. You should switch if the conditional expectation of $Y_2$ is greater than that of $Y_1$. We compute this conditional expectation mistakenly as follows, by further conditioning on whether $Y_1$ is the bigger or smaller envelope. Letting $E_1$ be the event that $Y_1$ is the bigger envelope, and $E_2$ the event that $Y_2$ is bigger, then $$ \begin{align} E[Y_2\mid Y_1] &\stackrel{\color{red}{!!}}= E\big[Y_2\mid Y_1,E_1\big]P(E_1)+ E\big[Y_2\mid Y_1,E_2\big]P(E_2) \\&=E\big[2Y_1\mid Y_1,E_1\big]\times \tfrac12+ E\big[\tfrac12 Y_1\mid Y_1,E_2\big]\times \tfrac12 \\&=(2Y_1)\times \tfrac12+ (\tfrac 12 Y_1)\times \tfrac12 \\&=\tfrac54 Y_1 \\&> Y_1 \end{align} $$ Since $E[Y_2\mid Y_1]>Y_1$ regardless of the value of $Y_1$, it would follow that it is always advantageous to switch envelopes, so you should switch even before opening envelope one, which clearly contradicts the symmetry of the problem.

The so-called paradox only arises because we mistakenly applied $(!)$ instead of $(\checkmark)$. If you make this correct substitution in $\stackrel{\color{red}{!!}}=$, then you will find that $ P(E_1\mid Y_1)$ is a complicated function of $Y_1$ depending on the distribution of $X$, which is sometimes bigger than $\frac12$ and sometimes less, so that depending on the value of $Y_1$ you see it is sometimes better to switch and sometimes worse. The paradox vanishes!