I'm investigating Gelfand-Pinsker channels where the capacity formula has been proven to be:
$$ C = max_{p(x, u|s)} [I(U; Y) - I(U;S)] .$$
I've written some simulation code that, given the distribution for a channel $W(Y|X,S)$ along with $p(x,u|s)$, can compute the mutual information above. To get the capacity, I need to find the maximum over all distributions $p(x,u|s)$. At the moment, I don't know an efficient way of doing this, so I generate random values to produce a $p(x, u|s)$, which contains $|X| \cdot |U| \cdot |S|$ entries. I can run the simulation many times over, each time generating a new $p(x, u|s)$ and I then find the max over all runs.
At the moment I'm only concerned with $|X| = |S| = 2$ but $|U|$ can be as large as I'd like, so my sample space dimension is growing by 4 for each increase in $|U|$. The problem is, as $|U|$ increases, the results become worse and worse and I need to use many runs of the simulation. This is really inefficient, already taking hours for millions of runs, and even then giving relatively poor results.
I'm wondering if there are some techniques I can use to more efficiently make such a calculation of channel capacity, or maybe just better methods of random sampling in general.
Thanks.
What I found is that I could use the DIT Python Library to calculate mutual informations quite easily for the channel for an arbitrary probability distribution $p(x, u|s)$. I wrote a function that takes a vector representing the distribution as input and outputs the MI give that distribution. I then used SciPy's optimization to find the minimum after I multiplied the out MI of the function by -1. Some restrictions are needed to the problem but SciPy's library allows for adding input restrictions. The results gave me what I expected in an efficient manner.