In the book Rationality: From AI to Zombies, in the chapter Absence of Evidence Is Evidence of Absence, Eliezer Yudkowsky (the author) wrote the following:
If $E$ is a binary event and $P(H|E) > P(H)$, i.e., seeing $E$ increases the probability of $H$, then $P(H|¬E) < P(H)$, i.e., failure to observe $E$ decreases the probability of $H$. The probability $P(H)$ is a weighted mix of $P(H|E)$ and $P(H|¬E)$, and necessarily lies between the two.
(By "$\neg E$" Eliezer means "$E^{\complement}$", and by "failure to observe $E$" he means "observing that $E$ is false". Similar terminology was discussed in another question (whose subject is very similar to the subject of this chapter))
According to the law of total probability, $P(H)=P(H|E)P(E)+P(H|\neg E)P(\neg E)$, so indeed as Eliezer (and wikipedia) noted, $P(H)$ is a weighted average of $P(H|E)$ and $P(H|¬E)$. Along with the assumption that $P(H|E) > P(H)$, we can deduce that $P(H|¬E)\le P(H)$.
So I see why $P(H|E)>P(H)$ implies $P(H|\neg E)\le P(H)$.
But does $P(H|E)>P(H)$ imply $P(H|\neg E)<P(H)$ ?
Is the following a valid counterexample, or am I missing something?
Let's say a real number in $[0,1]$ is chosen uniformly at random, let $H$ be the event that $1$ was chosen, and let $E$ be the event that an integer was chosen.
If I understand correctly:
- $E$ is a binary event.
- $P(H|E)=0.5$
(here is a question in which a similar calculation of a conditional probability is discussed) - $P(H)=0=P(H|\neg E)$
($P(H)$ is indeed a weighted average of $P(H|E)$ and $P(H|¬E)$, as $P(E)=0$) - Thus, $P(H|E)>P(H)$ is true, but $P(H|\neg E)<P(H)$ is false.
Characteristic for conditional probability $P(A\mid B)$ is the equality:$$P(A\mid B)P(B)=P(A\cap B)$$
Applying that here (i.e. multiplying both sides with $P(E)$), statement $P(H\mid E)>P(H)$ can be converted into:$$P(H\cap E)>P(H)P(E)\tag1$$Similarly statement $P(H\mid E^{\complement})<P(H)$ can be converted into:$$P(H\cap E^{\complement})<P(H)P(E^{\complement})$$or equivalently: $$P(H)-P(H\cap E)<P(H)-P(H)P(E)\tag2$$
It is obvious now that $(1)$ and $(2)$ are equivalent statements.
Your counterexample makes use of an event $E$ that satisfies $P(E)=0$ making it tricky to speak of conditionals like $P(H\mid E)$.
Characteristic for $P(H\mid E)$ is actually the equation $$P(E)P(H\mid E)=P(H\cap E)$$ and nothing more than that.
But in your case $P(E)=P(H\cap E)=0$ so that for any value for $P(H\mid E)$ this will be satisfied.
So there is no ground for setting $P(H\mid E)=0.5$ as you did.