I have a pair of dependent random variable $(\theta, X)$ where $\theta\in K$ for a compact subset $K\subset\mathbb{R}$ and $X\in\mathbb{R}^d$ with marginals $P_{\theta}$ and $P_X$.
I want to construct a random variable $Y$ from $X$ which is independent with $\theta$ and has the maximum mutual information with $X$.
The simplest case I am currently looking at is when $\theta\in \{0,1\}$ and $X=(X_1,X_2,\dots, X_d)$ and each $X_i\in \{1,2,\dots, n\}$.
Any idea on how to extract this random variable from $X$?