Are conditional probabilities being used correctly in this randomized detector matrix?

37 Views Asked by At

In Convex Optimization by Boyd there is a section on optimal detectors and hypothesis testing where the following equivalence for conditional probabilities are asserted.

P is an n x m matrix where:

$$ p_{kj} = prob( X = k | Q = j ).$$

Where X is a random variable with a distribution that depends on parameter Q.

(See Boyd §7.3, p. 364 bottom, here the greek letter theta has been swapped with Q to avoid potential formatting errors)

T is an m x n matrix where:

$$t_{ik} = prob( G = i | X = k )$$

Where G is the guess for the value of parameter Q given observation X = k. (See Boyd §7.3.1, p. 365)

They then define a detection probability matrix as:

$$D_{ij} =( TP )_{ij} = prob( G = i | Q = j )$$

(See Boyd §7.3.2, p. 366)

In other words the detector is making the calculation: $$ P( G = i|Q = j) = \sum_{k} P(G_i|X_k)*P(X_k|Q_j)$$

I may be making a very elementary error but my understanding is: $$ P( G = i|Q = j) = \sum_{k} P(G_i\bigcap X_k|Q_j)$$

And as $$ \sum_{k} P(G_i|X_k)*P(X_k|Q_j) \neq \sum_{k} P(G_i\bigcap X_k|Q_j)$$

I am unclear as to how this conclusion was reached. Is this just an estimate given a lack of knowledge? Boyd addresses it tangentially in a lecture as a distinction between a bayesian vs frequentist approach. However the book then goes on to state that in a Bayesian detector design we would minimize the probability of the error stated as: $$q^TP^e$$ where the probability of error = 1 - Probability that we guess correctly. i.e.

$$ P^e = 1 - D_{ii}$$ and $$q_i = P(Q =i)$$

This does not seem that this fully addresses the discrepancy as $$ P( G = i|Q = i) \neq \sum_{k} P(G_i|X_k)*P(X_k|Q_i)* q_i$$

I'm sure I must be doing something incorrectly in terms of calculating these conditional probabilities so any thoughts or insights would be appreciated thanks!

1

There are 1 best solutions below

1
On BEST ANSWER

The Law of Total Probability does state:

$\qquad\def\={{\,=\,}}\mathsf P(G\=i\mid Q\=j)=\sum_k\mathsf P(G\=i\mid X\=k, Q\=j)\,\mathsf P(X\=k\mid Q\=j)$

However, do note that $\mathsf P(G\=i\mid X\=k, Q\=j)=\mathsf P(G\=i\mid X\=k)$ (for all $i,j,k$) exactly when: $\boldsymbol G$ and $\boldsymbol Q$ are conditionally independent when given $\boldsymbol X$.

Your text should have made that statement clear, although it may be hidden in a Bayesian Directed Acyclic Graph or something.

Eg: The factorisation for $Q\to X\to G$ is:

$\qquad\mathsf P(G\=i, X\=k, Q\=j)=\mathsf P(G\=i\mid X\=k)\,\mathsf P(X\=k\mid Q\=j)\,\mathsf P(Q\=j)$


Anyway, when $G\perp Q\mid X$ we do have:

$\qquad\mathsf P(G\=i\mid Q\=j)=\sum_k \mathsf P(G\=i\mid X\=k)\,\mathsf P(X\=k\mid Q\=j)$

So, then $D_{ij} = \sum_k t_{ik}\,p_{kj}$