Upper bounding by 1 a sum of certain probability fractions

37 Views Asked by At

Introduction

Let $(X,Y)$ be a discrete joint probability distribution, letting $X$ model the rows of a matrix and $Y$ the columns. Overlaying this matrix sits a (discrete or continuous) probability distribution $Z$, such that its sample space is partitioned among the matrix cells $[i,j]$. Distribution $Z$ contains an event $E$.

A matrix cell is sampled with probability $P([i,j])>0$ and marginal probabilities are denoted $P([i,\cdot])$ and $P([\cdot,j])$, representing the probabilities of sampling row $i$ and sampling column $j$, respectively. We write $P(E\mid [i,j])$ for the probability of event $E$ occurring upon sampling $Z$ restricting to matrix cell $[i,j]$, and likewise for $P(E\mid [i,\cdot])$ and $P(E\mid [\cdot,j])$.

Note we have $$ \sum_i P([i,\cdot]) = \sum_j P([\cdot,j]) = 1,\ \ \ P([i,j]) = P([i,\cdot])\cdot P([\cdot,j]) $$

Statement

I hope to prove the upper bound $1$ for the following expression: $$ \sum_i P([i,\cdot]) \sum_j P([\cdot,j])\ \frac{P(E\mid [i,j])}{ P(E\mid[i,j])\cdot P([\cdot,j] + \sum_{k\neq j} P(E\mid[\cdot,k])\cdot P([\cdot,k]) } $$

In other words, summing over every matrix cell $[i,j]$ we use expressions $U,V,W$ listed below to compute $$\frac{U[i,j]}{V[i,j]+W[i,j]}$$

  • $U[i,j] = P(E\mid [i,j])\cdot P([i,\cdot])\cdot P([\cdot,j]) = P(E\mid [i,j])\cdot P([i,j])$

    The probability of $E$ occurring in cell $[i,j]$.

  • $V[i,j] = P(E\mid [i,j])\cdot P([\cdot,j])$

    The probability of $E$ occurring in cell $[i,j]$ conditioned on sampling from row $i$

  • $W[i,j] = \sum_{k\neq j} P(E\mid[\cdot,k])\cdot P([\cdot,k])$

    The probability of $E$ occurring outside column $j$.

I hope this problem is well formulated.

Example

The simplest non-trivial case is a $2\times2$ matrix. Writing $$ P(E\mid[1,1])=a,\ \ P(E\mid[1,2])=b,\ \ P(E\mid[2,1])=c,\ \ P(E\mid[2,2])=d \\ P([\cdot,1])=L\text{ (left)},\ \ P([\cdot,2])=R\text{ (right)},\ \ P([1,\cdot])=T\text{ (top)},\ \ P([2,\cdot])=B\text{ (bottom)} $$ the bound becomes $$ 1\geq \frac{a\cdot T\cdot L}{a\cdot L+(b\cdot T+d\cdot B)\cdot R} + \frac{b\cdot T\cdot R}{b\cdot R+(a\cdot T+c\cdot B)\cdot L} + \frac{c\cdot B\cdot L}{c\cdot L+(b\cdot T+d\cdot B)\cdot R} + \frac{d\cdot B\cdot R}{d\cdot R+(a\cdot T+c\cdot B)\cdot L} $$ See this expression plotted with $L=0.6,\ R=0.4,\ T=0.9,\ B=0.1$ and $a\in[0,1],\ b=0.126,\ c=0.166,\ d=0.093$.

A simpler expression for the example (but with more complex variable bounds) can be had setting $a'=a\cdot T\cdot L,\ b'=b\cdot T\cdot R,\ c'=c\cdot B\cdot L,\ d'=d\cdot B\cdot R$ yielding $$ 1\geq \frac{a'}{a'/T+(b'+d')} + \frac{b'}{b'/T+(a'+c')} + \frac{c'}{c'/B+(b'+d')} + \frac{d'}{d'/B+(a'+c')} $$

How to prove?

Inequalities like Hölder's, Jensen's, and AM-GM don't seem immediately applicable. I'd appreciate insight on proving strategies. Can this expression be interpreted as some probability in some distribution, and the bound follows? I hope to see this proven or see a counterexample.

1

There are 1 best solutions below

0
On

We first reduce to a simpler version of the following inequality for each $j$, and then apply Jensen's inequality. $$ \begin{align} &\sum_i \frac{P(E\mid [i,j])\cdot P([i,\cdot])\cdot P([\cdot,j])}{ P(E\mid[i,j])\cdot P([\cdot,j]) + \sum_{k\neq j} P(E\mid[\cdot,k])\cdot P([\cdot,k]) } \\ &\leq \sum_i \frac{P(E\mid [i,j])\cdot P([i,\cdot])\cdot P([\cdot,j])}{ P(E\mid[\cdot,j])\cdot P([\cdot,j]) + \sum_{k\neq j} P(E\mid[\cdot,k])\cdot P([\cdot,k]) } \end{align} $$ The denominator of the upper bound simplifies as $$ \sum_{k} P(E\mid[\cdot,k])\cdot P([\cdot,k]) = P(E) $$ Summing over $j$ then yields $$ \sum_j \sum_i \frac{P(E\mid[i,j])\cdot P([i,j])}{P(E)} = \sum_j \frac{P(E\mid[\cdot,j])\cdot P([\cdot,j])}{P(E)} = \frac{P(E)}{P(E)} = 1 $$

To simplify the inequality above, we first divide by $P([\cdot,j])$ which is non-zero by assumption, and then defining $x\geq0$ as $$ x = \sum_{k\neq j} P(E\mid[\cdot,k])\cdot P([\cdot,k]) \big/ P([\cdot,j]) $$ the inequality becomes $$ \sum_i \frac{P(E\mid [i,j])\cdot P([i,\cdot])}{ P(E\mid[i,j]) + x } \leq \sum_i \frac{P(E\mid [i,j])\cdot P([i,\cdot])}{ P(E\mid[\cdot,j]) + x } $$ With $j$ fixed and irrelevant to the inequality we rename as $$ y_i = P(E\mid[i,j]),\text{ constrained to } y_i\in[0,1] \\ w_i = P([i,\cdot]),\text{ constrained to } w_i\in(0,1],\ \sum_i w_i = 1 \\ $$ With $$ P(E\mid[\cdot,j]) = \sum_l P(E\mid[l,j])\cdot P([l,\cdot]) = \sum_l y_l\cdot w_l $$ we have $$ \sum_i \frac{y_i\cdot w_i}{y_i+x} \leq \sum_i \frac{y_i\cdot w_i}{\left(\sum_l y_l\cdot w_l\right) + x} = \frac{\sum_i y_i\cdot w_i}{x + \sum_l y_l\cdot w_l} $$

To prove this inequality we apply Jensen's inequality for the function $f(z)=z/(z+x)$ which is concave for $z\geq0$ since $x\geq0$. Jensen's inequality stated below translates to the desired inequality. $$ \sum_i f(y_i)\cdot w_i \leq f\left(\sum_i y_i\cdot w_i\right) $$