Probability of two events with mutually exclusivity among them adding to 1

Question

Okay, I thought I had this, but it seems I have no idea.

We have two events, A and B. And each event has 3 "states". We will call these states 1,2,3. And we have the following rules:

• For each event, states 1,2 and 3 are mutually exclusive. This means that for example $P(A1)+P(A2)+P(A3) = 1$

So, if I start by considering one event only, and I have $P(A1) = 0.5$ and $P(A2) = 0.3$, I can deduce $P(A3) = 0.2$.

However I now want to add this rule:

• Among the two events, if A is in state 2, B cannot be in state 2. This means, state 2 is mutually exclusive with the other event.

And I'm lost completely. I'll try to put an example with dies to see if I can explain myself better. Event A will be the first dice, Event B will be the second dice. These dice are special, they only have 3 numbers in them, these being 1, 2, and 3.

Now, If I take each dice separately, I know the probabilities. For example, for dice A, I only have three possibilities, it is either a 1, a 2 or a 3. And these are mutually exclusive. However, if I now take into consideration that if one dice gets a state 2 the other dice magically cannot get a state 2, how would I get the probability of getting a sum of 4?

If there was not that second rule, it would be:

$P(A1\cap B3) + P(A2\cap B2) + P(A3\cap B1)$

This means, all the states possible that result in a 4. However, now I don't have $P(A2\cap B2)$, because it is $0$.

I'm gonna put some numbers. Charged dies, yeah.

• P(A1) = 0.5, P(A2) = 0.2, P(A3) = 0.3
• P(B1) = 0.5, P(B2) = 0.2, P(B3) = 0.3

Now, what would be the probability of getting a $4$? If I use the previous equation but simply putting a 0 in the forbidden combination

$P(4) = 0.5\times 0.3 + 0 + 0.3\times 0.5 = 0.30$

However, the whole search space of addition are 2,3,4,5,6. And they do not add up to 1.

$P(2) = P(A1\cap B1) = 0.5\times 0.5 = 0.25$ $P(3) = P(A1\cap B2) + P(A2\cap B1) = 0.5\times 0.2 + 0.2 \times 0.5 = 0.20$ $P(5) = P(A2\cap B3) + P(A3\cap B2) = 0.2\times 0.3 + 0.3 \times 0.2 = 0.12$ $P(6) = P(A3\cap B3) = 0.3\times 0.3 = 0.09$

The sum gives: $P(2+3+4+5+6) = 0.96$. What is lacking? Correct, $P(A2\cap B2) = 0.2\times 0.2 = 0.04$.

How would the corect modified probabilities be? What would be the correct probability of getting a sum of 4? I'm totally lost :c

--- UPDATE ----

Thank you very much for the answers. I really got more insight in all this conditioned probability thingy and probability spaces. If I could get a last help here it would be much appreciated.

The real simplified case I'm facing is something like, we have two machines, $1$ and $2$. They have three possible states, working ($W$), maintenance ($M$) and repair ($R$). When they work by themselves, only one at a time, it follows the rule:

(1) Cannot be working and in maintenance or repairing at the same time. Cannot be in maintenance and working or repairing at the same time. Cannot be repairing and in maintenance or working at the same time.

This rules gives me a probability space of $P(W) + P(M) + P(R) = 1$. And I have these numbers, things like:

• P(W1) = 0.6, P(M1) = 0.3, P(R1) = 0.1
• P(W2) = 0.5, P(M2) = 0.2, P(R2) = 0.3

The problem is when I try to map the space of the two machines. I have to add this new rule:

(2) If machine 1 is in maintenance, then machine 2 will never be in maintenance, and viceversa.

The rule is intuitive, maintenance is a state that you can program, so it would be obvious to not want make both maintenances at the same time. The complete space would be now:

• $P(\text{one of the two works}) + P(\text{both work}) + P(\text{none work}) = 1$

I'm totally lost to how calculate any of these. I don't really know which events are independent here and which are not, or if I should scale things like told.

Sorry for being this hardheaded on this matter, I really want to understand better probability.

Answer 1

You're modeling the events as unconditional but introduced a condition (one outcome is impossible). The general conditional probability formula is: $P(B|A) = P(A$ and $B) / P(A)$. In this problem, we will only need to find the probability of each outcome (each 3-sided dice sum) given $A2 \cap B2$ cannot occur. These conditional probabilities are what will sum to 1. Ultimately we'll use the general conditional probability formula to effectively reduce our probability space to $1-P(A2 \cap B2)=P((A2 \cap B2)')=0.96$ due to the condition.

First, let's apply the conditional probability rule with a simple example using a 6-sided die (values are 1 though 6). We ask, what is the probability of rolling a 2 given we know the result is even? We write this using our general conditional probability formula: $P(2|E)=P(2$ and $E)/P(E)$. So $P(2$ and $E)$ is simply the probability of rolling a 2 and having the number be even. This is the same as $P(2)=1/6$ as every 2 we roll will be even. $P(E)=1/2$ so we're left to conclude that $P(2|E)=(1/6)/(1/2)=1/3$ which is the intuitive result. This type of condition only had the effect of shrinking the probability space from 6 to 3 possibilities (the 3 even values).

Now the case stated where the event of interest is the sum of 2 3-sided dice with values 1 through 3 of varying probabilities. As stated in the question, the (unconditional) probability of a sum of 2 is 0.25. We now ask, what is the probability of a sum of 2 given (conditional on) the probability of rolling two 2's concurrently (in the same trial) is set to zero. Instead of the unconditional $P(S2)=0.25$, where S2=a sum of 2 result, we're interested in $P(S2|i)$ where $i$ is the known impossibility of both dice rolling 2 in the same trial (we could think of $i$ as "impossibility" or "new information"). At first this may not seem to impact the event $S2$ but in fact we're reducing the possible outcomes of our trials just the same as reducing the probability space to only include even results in the previous example.

Plugging into the conditional probability formula we see that $P(S2|i)=P(S2$ and $i)/P(i)$

$P(S2$ and $i)=P(S2)=0.25$ as the special property is applied to the dice already in the same way that adding the "even" condition in the previous simple example had no impact on the numerator of the conditional probability formula.

$P(i)=0.96$ the condition $i$ represents 96% of all possible unconditional outcomes in the same way that the "even" condition represented 50% of all possible outcomes in the simple example.

$P(S2|i)=P(S2$ and $i)/P(i)=P(S2)/P(i)=0.25/0.96 \approx 0.26$

When we conduct the same operation to all conditional events the sum is 1:

$P(S2|i)=0.25/0.96 \approx 0.260$

$P(S3|i)=0.20/0.96 \approx 0.208$

$P(S4|i)=0.30/0.96 \approx 0.313$

$P(S5|i)=0.12/0.96 \approx 0.125$

$P(S6|i)=0.09/0.96 \approx 0.094$

Intuition: An intuitive way of thinking of this is that you decreased the probability/possibility space by removing a possible event, thus you need to scale the individual event probabilities by dividing by 0.96 (think "out of the 96% of space we have left").

Alternate Intuition: Instead of imposing a condition on our dice we can instead imagine truncating observations already made to achieve the same result. Imagine just running the experiment unconditionally 100 times (rolling both dice, recording the sum) and the results aligned exactly to the probabilities (if it helps, imagine we ran this trial ad-infinitum). We would record the sum of each trial and would expect 25 (25%) of trials to result in a sum of 2. If we then deleted 4 (4%) of our data/trials (the sums of 4 resulting from $A2 \cap B2$ results) the remaining trials resulting in a sum of 2 would represent $25/96 \approx 26$% of the trials in our truncated (condition/filter-applied) dataset.

Answer 2

First of all, you really should upvote (or accept) the answer by Ron because he spent so much time on it -- and because he's correct. :) Instead here I will present a much simpler example and see if this helps.

You have two coins, one Gold and one Silver. Let $H_G$ denote the Gold coin showing Head, and similarly for $T_G, H_S, T_S$.

Suppose you believe the following to be true:

(1) The Gold coin is fair, i.e. $P(H_G) = P(T_G) = 1/2$.

(2) The Silver coin is fair, i.e. $P(H_S) = P(T_S) = 1/2$.

(3) The two coins are independent. This has a precise mathematical definition, i.e. $P(H_G, H_S) = P(H_G) P(H_S)$ and similarly for all other combos.

(4) Both coins will never show Tails together, i.e. $P(T_G, T_S) = 0$.

What we're saying is that the 4 beliefs are mutually mathematically inconsistent. You cannot have all 4 true at the same time. You can however, have any 3 out of 4 true at the same time. E.g.

(1+2+3): This is just your regular two fair, independent coins. $P(T_G, T_S) = 1/4 \neq 0$, violating (4).

(1+2+4): The two coins are linked (magically? magnetically?) so that they always show different results, i.e. $P(H_G, T_S) = P(T_G, H_S) = 1/2$, while $P(H_G, H_S) = P(T_G, T_S) = 0$. You can easily verify that if you look at just one of them, it looks fair (it is fair). It is also clear that this violates (3).

(1+3+4): The Gold coin is fair while the Silver coin shows Head all the time. You can verify that this satisfies the equations defining independence.

(2+3+4): Similar to above.

These are 4 different models (probability spaces, alternate universes). All 4 of them are completely valid, albeit different, probability spaces.

Unfortunately for you, there just isn't a probability space (alternative universe) where all 4 beliefs are true. If you insist on believing all 4, and use the math accordingly, you will get wrong results such as:

$$P(H_G, H_S) + P(H_G, T_S) + P(T_G,H_S) + P(T_G,T_S) = {1\over 2}\cdot{1 \over 2} + {1\over 2}\cdot{1 \over 2} + {1\over 2}\cdot{1 \over 2} + 0 = {3 \over 4}$$

when of course the sum of probabilities need to equal $1$. What caused the problem? The root cause is your insistence that all 4 beliefs are true. You can easily verify that, if you drop any one of the beliefs, i.e. if you use any of the 4 valid models, in each case the sum of probabilities is $1$.

---

None of the above concern conditional probability at all. Instead, conditional probability comes in when you first fix a probability space, then make an observation (which in a sense "shrinks" the space and "rescales" the probabilities). E.g.

(5) You throw the coins without looking at the result. Your friend looked, and then truthfully told you that they are not both Tails.

In each of the 4 models, you can condition on (5).

In the (1+2+4), (1+3+4) or (2+3+4) models, conditioning on (5) doesn't change a thing. After all, if one of the coins always shows Head, or if they are always different, then of course they won't both show Tails. Your friend hasn't told you anything you didn't already know.

In the (1+2+3) model, conditioning on (5) "changes" the probability values. I put "changes" in quote because technically the "before" value is called prior and the "after" value is called posterior and they are different mathematical objects. E.g. "before" your friend told you, you have $P(T_G) = 1/2$, the prior value. But "after" your friend told you, you have $P(T_G | not(T_G,T_S)) = 1/3$, the posterior value. This is calculated via:

$$P(T_G | not(T_G,T_S)) = {P(T_G \cap not(T_G,T_S)) \over P(not(T_G,T_S))} = {P(T_G,H_S) \over 1 - P(T_G,T_S)} = {1/4 \over 1 - 1/4} = {1 \over 3}$$

Note that in the very calculation itself, we used the prior value $P(T_G,T_S) = 1/4$. We never ever pretend this prior value is $0$. We condition on it not happening this time, but we never said its prior probability is $0$.

Having $(1 - P(T_G,T_S))$ as denominator is what Ron meant by shrinking/rescaling the space.

Note also that using the (1+2+3) model and conditioning on (5), is not equivalent to any of the (1+2+4), (1+3+4), (2+3+4) models. After all, the value $1/3$ never occurs in any of those models.

Note also that, since $P(T_G | not(T_G,T_S)) = 1/3$, it is no longer 50-50 ("fair") odds that the Gold coin shows Tail. But the coin is supposed to be fair, so what does $1/3$ mean? It means this:

Imagine the coins are actually independent and fair, and you throw them 1 billion times, but at each trial, before you get to look, your friend looks first and if both Tails, he picks up the coins, gives them to you, and tells you that trial doesn't count and you have to do over. Then, after 1 billion trials (excluding the do-overs), the Gold coin will only show Tail about $1/3$ of the time. If these are the only Gold coins you ever see, then for all practical purposes for you, the coin is not fair. In fact you would conclude that the coins are not only not fair, but also not independent, and specifically that they are (magically? magnetically?) linked so that

$$P(H_G, H_S) = P(H_G, T_S) = P(T_G, H_S) = 1/3$$

BTW this last equation also describes a valid probability space -- it is a 5th model which is different from all the other 4 models we've discussed so far. This probability space does satisfy your belief (4) $P(T_G,T_S) = 0$, but it in fact violates all your other beliefs (1), (2), and (3).

Hope I'm helping as opposed to making this even more confusing... :)

Answer 3

Starting a new answer because this addresses specifically the updated question.

You have two machines, each of which can be in one of $3$ (disjoint, i.e. non-overlapping) states. So there are $9$ combos total: $W_1 W_2, W_1 M_2, W_1 R_2, M_1 W_2, M_1 M_2, M_1 R_2, R_1 W_2, R_1 M_2, R_1 R_2.$ These are your $9$ sample points. The collection of all $9$ is your sample space.

Technically, probabilities are assigned to events (subsets of the sample space), but for a discrete, finite space like yours, it is sometimes better to think of probabilities being assigned to individual sample points. I.e., you get to specify $9$ numbers, $P(W_1 W_2), P(W_1 M_2),$ etc. You can specify these any way you want, subject to two basic rules: First, they must all be between $0$ and $1$, and second, they must sum to $1$. You have $9$ unknowns and $1$ equation (they must sum to $1$), so you have $8$ degrees of freedom.

Put your $9$ values into a matrix in the obvious way:

$$\begin{bmatrix} P(W_1 W_2) & P(M_1 W_2) & P(R_1 W_2) \\ P(W_1 M_2) & P(M_1 M_2) & P(R_1 M_2) \\ P(W_1 R_2) & P(M_1 R_2) & P(R_1 R_2) \\ \end{bmatrix}$$

In your case you want e.g. $P(W_1) = 0.6$. Presumably, this is to be interpreted as you wanting: $P(W_1 W_2) + P(W_1 M_2) + P(W_1 R_2) = 0.6$, i.e. the first column sums to $0.6$. This gives another equation and eliminates one degree of freedom.

You have one equation for every row, and one for every column. However, some of them are redundant, e.g., if the first column sums to $0.6$ and the second column sums to $0.3$, then the third column must sum to $0.1$ since the entire matrix sums to $1$. In short, even though you have $6$ equations, plus another one saying the matrix must sum to $1$, you actually only have $5$ constraints for these $9$ values. Thus you still have $4$ degrees of freedom.

Finally you also want $P(M_1 M_2) = 0$. You still have $3$ degrees of freedom. So your problem is under-specified and there are a lot of solutions! E.g. one way (out of many) to show the $3$ degrees of freedom is that, for any $a,b,c$, the following matrix meets all your requirements (as long as all entries remain within $[0,1]$):

$$\begin{bmatrix} 0.5 -a-b & 0+b & 0 +a\\ 0.1 +a+c & 0 & 0.1 -a-c \\ 0 +b-c & 0.3-b & 0+c\\ \end{bmatrix}$$

You can easily check that each column/row sum is your required value. Think of the case of $a=b=c=0$ as the "base" solution and then you can just adjust $a,b,c$ to "squish" values around the matrix, while maintaining all the column/row sum constraints.

Now how should we decide $a,b,c?$ You didn't tell me! So I have no idea. :)

Actually I have some guess. If not for the requirement that the middle value $P(M_1 M_2) = 0$, a very natural alternate and incompatible requirement is that the machines are independent. This means $P(W_1 W_2) = P(W_1) P(W_2) = 0.6 \times 0.5 = 0.3$ and similarly for the other $8$ combos. This actually introduces $9$ new equations, but they are very redundant with many of the other equations, so that in the end you get these $9$ numbers:

$$\begin{bmatrix} 0.3 & 0.15 & 0.05 \\ 0.12 & \mathbf{0.06} & 0.02 \\ 0.18 & 0.09 & 0.03 \end{bmatrix}$$

Alas, the middle number is $P(M_1 M_2) = P(M_1) P(M_2) = 0.3 \times 0.2 = 0.06$, not the $0$ that you want.

One way to make the middle $0$ is to just change it to $0$, and then since the matrix must still sum to $1$, so you rescale everybody else by ${1 \over 1 - 0.06}$. This is equivalent to "conditioning on $not(M_1 M_2)$" in the original (independent machines) model, but as you can see, in the final result all the column/row sums are no longer what you want. I.e. the new sum for the first column $= P(W_1 | not(M_1 M_2)) = 0.6/0.94 \approx 0.638$. If you find this acceptable, this may be a good approach. After all, you're only off by $6$%.

OTOH, if you must maintain all the column/row sums, e.g. first column must still sum to $0.6$ exactly, then I don't know of any theoretically justifiable way to do this. Still, a "hand-crafted" attempt can be made by forcing the middle to $0$, while redistributing and disturbing the rest of the matrix "as little as possible". E.g. with a little trial and error I found:

$$\begin{bmatrix} 0.3-0.015 & 0.15+0.03 & 0.05-0.015 \\ 0.12+0.03 & 0 & 0.02+0.03 \\ 0.18-0.015 & 0.09+0.03 & 0.03-0.015 \end{bmatrix}$$

In this new matrix, the $9$ equations for independence no longer hold, but you gotta give up something. Also, it should be obvious there are many, many other ways to "squish" and redistribute the $0.06$ around, some more dramatic than others. The above is just one example where I made some vague attempt to change the other numbers "as little as possible" compared to the independent machines model. But then "as similar as possible to the independent model" might not be a good criterion. Perhaps you have other "real world inspired" criterion in mind?

In short, it would be nice to keep the independence equations, but they are by definition not compatible with the middle number being $0$. And once you give up the independence equations, you have lots of freedom which is both a blessing (you can almost do whatever you want!) and a curse (you cannot justify what you end up doing, nor find an easy way to remember how to calculate each entry). And you may have other "real world inspired" criterion in mind, but until you can clearly articulate it, that doesn't help with the math.

BTW, once you have finally decided on one matrix (using whatever aesthetic criteria), you will have no problem calculating e.g. $P(none\ work) = P(M_1 M_2) + P(M_1 R_2) + P(R_1 M_2) + P(R_1 R_2)$. But I don't think you can calculate that without first deciding which exact matrix you want.

---

Postscript: actually, here is one attempt at a "real world inspired" criterion. You said $P(M_1 M_2) = 0$ because in real life both maintenance would not be scheduled together. But if machine $1$ needs to be maintained $0.3$ of the time, $P(M_1) = 0.3$, and machine $2$ needs to be maintained $0.2$ of the time, $P(M_2) = 0.2$, and they cannot be scheduled together, then how about scheduling them separately during otherwise normal "working" hours? I.e. in order to avoid any $M_1 M_2$ time, we take some $W_1 W_2$ time and turn it into maintaining one of the machines instead. Mathematically:

$$\begin{bmatrix} 0.3-0.06 & 0.15+0.06 & 0.05 \\ 0.12+0.06 & \mathbf{0} & 0.02 \\ 0.18 & 0.09 & 0.03 \end{bmatrix}$$

Incidentally, another "real world inspired" criterion also leads to this same matrix, namely: "machine breakdowns (and therefore repairs) cannot be controlled by anybody." This can be interpreted mathematically as: $R_1$ is independent of all other events, and $R_2$ is also independent of all other events. This means the independence equations hold for the $5$ values in the bottom row and rightmost column - so they must have these values shown. Then you also mandate the middle value to be $0$. Finally you still have the row/column sum constraints, which mandate the remaining $3$ values in the "top left triangle". Thus the above matrix is the only matrix satisfying all your original constraints plus this new "real world inspired" critierion of $R_1$ and $R_2$ being independent of everything else.

Only you can tell me if this "real world inspired" modification to the matrix bears any semblance to your actual real world application. :) I just wanna point out though: even if you end up liking this model, it's just your choice, and there is nothing in probability theory that prefers this over many other choices (unless you add these "real world" criterion as specifications).