I’m developing a tool to calculate the probability of conclusions using facts and Bayes Rule. I’ve encountered quite a puzzle. The problem is that when I use two different forms of Bayes Rule and estimated variables I get two different results. Let me try to explain.
Form A - The standard form of Bayes Rule is:
P(conclusion | fact) = P(conclusion) x P(fact | conclusion) / P(fact).
Form B - P(fact) must be based on prior data. If P(fact) is not available, then the no prior data form of Bayes Rule can be used. This is:
P(conclusion | fact) = P(conclusion) x P(fact | conclusion) / (P(conclusion) x P(fact | conclusion) + (1 – P(conclusion)) x P(fact | not conclusion))
Let’s use an example. The fact (the effect) is that a graph shows a clear national GDP growth long-term trend after a recession was quickly corrected. The conclusion (the cause) is the long-term GDP growth trend is due to good management of economic policy. This gives us five variables of interest. All but the first are estimated.
P(conclusion | fact) = unknown. This is what we want to calculate.
P(conclusion) = .01. We start with a low prior probability so that we can accumulate evidence that P(conclusion | fact) is highly likely to be true.
P(fact) = .95. All data points to a clear strong trend. No data is contested.
P(fact | conclusion) = .99. The probability rises from .95 to .99 given the conclusion.
P(fact | not conclusion) = .05. If the conclusion is false, then the fact is thrown into high doubt. It somehow cannot be true at all. .05 represents extremely unlikely.
Plugging these five variables into the two forms of Bayes Rule, we get
Form A: .01 x .99 / .95 = .01
Form B: .01 x .99 / (.01 x .99 + ((1 - .01) x .05)) = .167
The two results, .01 and .167, do not agree. They are not even close. Why is this?
The discrepancy can be eliminated two ways, by changing estimates to calculated values using the other variables.
The first way to eliminate the discrepancy is to change P(fact | not conclusion) from .05 to .95. But that makes no sense. .95 says the fact is still highly true. But it cannot be highly true if the conclusion is false. A false conclusion would say “the long-term GDP growth trend is NOT due to good management of economic policy.” Therefore, it must be due to something else. What would that be? BAD economic policy? Luck? Sudden stimulus due to war? None of the alternative reasons make any sense, because in this case (the reaction by the US administration to the 2008 Great Recession) they did not occur, though that could be controversial. What I’m trying to explain is that estimates for P(fact | not conclusion) for this example will tend to be low.
The second way to eliminate the discrepancy is to change P(fact) from .95 to .06. But that also makes no sense. It says the fact is almost completely false. But the data trend is clear and uncontested. There’s no way it can be false.
Of interest is that when the two forms of Bayes Rule are used on problems where the variables can be set by counting in a sample space, there is no discrepancy. This makes me suspect that Bayes Rule works well for sampling logic, but does poorly for some types of general logic, where some variables must be estimated.
Form A’s result make little sense. The prior and the posterior probabilities are the same. That’s like ignoring an extremely useful fact. Reducing the estimate for P(fact) to increase the posterior doesn’t make sense, because that implies the less true the fact, the more true the conclusion.
Form B’s result makes the most sense. It agrees with intuition. Is this because when using general logic to estimate variables, general logic works best with form B and not form A?
Thanks.
P(FACT) =(P(conclusion) x P(fact | conclusion) + (1 – P(conclusion)) x P(fact | not conclusion)) by law of totaly probability https://en.m.wikipedia.org/wiki/Law_of_total_probability so it is not 0.95