Deriving individual probability from conditional probability

75 Views Asked by At

Very simple question - say you have discrete variables $a \in S$ and $x \in Q$, where $S$ and $Q$ are finite sets. Say also you know the value of $P(a|x)$ for all $S$ and $Q$. How do you derive $P(a)$ for all $a \in S$? I'm thinking the formula is something like the following:

$P(a) = \frac{\sum_{x \in Q}P(a|x)}{|Q|}$

Is this correct? What would the formula be if $Q$ were continuous?

1

There are 1 best solutions below

1
On BEST ANSWER

If $A$ and $X$ are random variables and we know $X$ takes values in a discrete set $\mathcal{Q}$ then for any real number $a$ we have by the law of total probability: $$ \boxed{P[A=a] = \sum_{x \in \mathcal{Q}} P[A=a|X=x]P[X=x]} \quad (1) $$ In the special case when $\mathcal{Q}$ is a finite set with $|\mathcal{Q}|$ elements, and when $X$ takes values equally likely over all elements of $|\mathcal{Q}|$, then $P[X=x] = \frac{1}{|\mathcal{Q}|}$ for all $x \in \mathcal{Q}$ and the above formula (1) reduces to: $$ P[A=a] = \sum_{x \in \mathcal{Q}} P[A=a|X=x]\frac{1}{|\mathcal{Q}|}$$ which is similar in form to your conjectured formula.

If $X$ is a continuous random variable with PDF $f_X(x)$ then the law of total probability formula (1) is changed to: $$ \boxed{P[A=a] = \int_{-\infty}^{\infty} P[A=a|X=x]f_X(x)dx} $$


Note also that we can only take probabilities of events. So if $Y$ is a random variable, example events are $\{Y\leq 12\}$ or $\{Y=8\}$ and we can speak of $P[Y\leq 12]$ (the probability that $Y$ is less than or equal to 12) and $P[Y=8]$ (the probability that $Y$ is 8) but it makes no sense to speak of $P[Y]$ (the probability that $Y$ is...what???).

So I do not like your notation $P[a]$ and $P[a|x]$ since (i) I do not know if $a$ is supposed to be a random variable or a parameter; (ii) $a$ certainly is not an event so $P[a]$ makes no sense (the probability that $a$ is...what???)