What is one way to express conditional distributions using influence functions?

51 Views Asked by At

Suppose I have a function defined as follows:

$$h: \{-1,1\}^n \to \{-1,1\}$$

Let $A$ be a fixed direction of the input domain ($A$ can be interpreted as some feature in the input domain $\{-1,1\}^n$), $A$ take +1 and -1 values, then $A = A^{+} \cup A^{-}$. We say that $x \in A^{+}$ when the variable $A$ takes +1 as a value, and similarly for $A^{-}$.

We denote $x^{\sim A}$ the operation of flipping the $A$ bit of $x$ (if the $A$ bit of $x$ is +1 then the A bit of $x^{\sim A}$ is -1). The influence function of the function h with respect to $A$ is defined as follows;

$Inf_A(h) = \mathbb{P}[h(x^{\sim A}) \neq h(x)]$

Is it possible to express the following property in terms of influence function? $\mu(h) = \Bigg|\mathbb{P}[h(x) = 1 |x \in A^{+}] - \mathbb{P}[h(x) = 1 |x \in A^{-}]\Bigg|$

Update:

Here's my attempt to solve this problem:

By definition

$Inf_A(h) = \mathbb{P}[h(x^{\sim A}) \neq h(x)]$

Denoting $p$ the probability mass of $A^{+}$ and expending the right term of the equation gives:

$Inf_A(h) = p \mathbb{P}\Big[h(x) = 1, h(x^{\sim A}) = 0 | x \in A^{+} \Big] + (1-p) \mathbb{P}\Big[h(x) = 1, h(x^{\sim A}) = 0 | x \in A^{-} \Big] + p \mathbb{P}\Big[h(x) = 0, h(x^{\sim A}) = 1 | x \in A^{+} \Big] + (1-p) \mathbb{P}\Big[h(x) = 0, h(x^{\sim A}) = 1 | x \in A^{-} \Big]$

Now let's make the assumption that the distribution over x is invariant by the action $\sim^A$.

The previous expression becomes:

$Inf_A(h) = p \mathbb{P}\Big[h(x) = 1, h(y) = 0 | x \in A^{+}, y \in A^{-} \Big] + (1-p) \mathbb{P}\Big[h(x) = 1, h(y) = 0 | x \in A^{-}, y \in A^{+} \Big] + p \mathbb{P}\Big[h(x) = 0, h(y) = 1 | x \in A^{+}, y \in A^{-} \Big] + (1-p) \mathbb{P}\Big[h(x) = 0, h(y) = 1 | x \in A^{-}, y \in A^{+} \Big]$

With the events being i.i.d, the expression becomes:

$Inf_A(h) = p \mathbb{P}\Big[h(x) = 1 | x \in A^{+} \Big] \mathbb{P}\Big[h(y) = 0 |y \in A^{-} \Big] + (1-p) \mathbb{P}\Big[h(x) = 1| x \in A^{-} \Big]\mathbb{P}\Big[h(y) = 0 | y \in A^{+} \Big] + p \mathbb{P}\Big[h(x) = 0 | x \in A^{+} \Big]\mathbb{P}\Big[h(y) = 1 | y \in A^{-} \Big] + (1-p) \mathbb{P}\Big[h(x) = 0| x \in A^{-}\Big]\mathbb{P}\Big[ h(y) = 1 | y \in A^{+} \Big]$

This gives us the simplified expression:

$Inf_A(h) = \mathbb{P}\Big[h(x) = 1 | x \in A^{+} \Big] \mathbb{P}\Big[h(x) = 0 |x \in A^{-} \Big] + \mathbb{P}\Big[h(x) = 1| x \in A^{-} \Big]\mathbb{P}\Big[h(x) = 0 | x \in A^{+} \Big] $

Which gives:

$$Inf_A(h) = \mathbb{P}\Big[h(x) = 1 | x \in A^{+} \Big] + \mathbb{P}\Big[h(x) = 1 |x \in A^{-} \Big] -2 \mathbb{P}\Big[h(x) = 1| x \in A^{-} \Big]\mathbb{P}\Big[h(x) = 1 | x \in A^{+} \Big] $$

Which is an expression of the form $a+b-2ab$ This gives us the final expression:

$Inf_A(h) = \mu(h)^2 - 2\sqrt{ab}(1 - \sqrt{ab} )$

Is there a better factorization to obtain a relationship between $Inf(h)$ and $\mu(h)$?