Could someone help me mathematically describe the following problem.
I have a vector $X = [x_1, ..., x_n]^T$ that represents a list of features. I also have a prediction function $p$ so that $p(X) = \lambda$ where $\lambda$ is the predicted class.
I have another vector $E = [\varepsilon_1, ..., \varepsilon_n]^T$ that contains the importance value of each corresponding feature of $X$.
What I'm trying to mathematically describe is that: if I generate a new duplicate vector $X^*$ that only keeps intact the features of $X$ that have the highest importance values in $E$, and randomly vary (i.e. add noise to) the remaining features (i.e. the ones that have low importance values), I should still observe that $p(X) = \lambda = p(X^*).$
I hope that makes sense.
Well, the details I could think to add might look like this:
Choose a $\varepsilon_{\text{cutoff}}$ that represents the cut-off importance: importances above this you'll keep, and those below you'll vary.
Choose your $X^*=[x_1^*, x_2^*,\dots,x_n^*]^T$ as follows: $$x_j^*=\begin{cases}x_j,\quad& \varepsilon_j\ge\varepsilon_{\text{cutoff}} \\ x_j+\theta_j, & \varepsilon_j<\varepsilon_{\text{cutoff}}\end{cases}. $$ Here $\theta_j$ is a random noise function, such as a Gaussian or (probably better) from a suitably chosen uniform distribution.
Evaluate $p(X^*)$ and check that your model predicts the same class as $p(X)$ does.
Repeat Steps 1-3 until you are satisfied that the new restricted features correctly predict the output class $\lambda$ regardless of whatever noise you introduce in the less important features.