Suppose I have a random vector $\bar{X}=[X_{1},X_{2}]$.
$X_{1}$ and $X_{2}$ comes from the alphabet {0,1,2}. I don't have any information on the probability distribution of these random variables. In fact, I am interested in finding the joint distribution of $X$.
I need to find the pdf of $\bar{X}$ which is the joint pdf of $X_{1},X_{2}$ such that
$-\sum$ $P_{\bar{X}}(X_{1},X_{2})$log $P_\bar{X}(X_{1},X_{2}$) is maximized.
All I have is a constraint:
$ E||U\bar{X}||^{2} <\alpha$ , where U is a $2 \times 2 $ zoom DFT matrix. That is, I am taking 2 samples in my frequency band of interest and limiting its power to be within a tolerance. (Not uniformly across the entire band)
Can anyone give some detailed explanation on how to go about this?
I am in fact interested in knowing how the solution scales if I have more elements (say N) in the random vector instead of 2.
If we impose the restriction $ E (||X||^2) = \alpha$ then we can try the standard Lagrage multipliers approach
$$g(p_{i,j}) = \sum_{i,j}(i^2 + j^2)p_{i,j} -\alpha=0 \tag{1}$$ $$h(p_{i,j}) =\sum_{i,j}p_{i,j} -1=0 \tag{2}$$
$$L(p_{i,j})= -\sum_{i,j} p_{i,j} \log p_{i,j} + \beta g(p_{i,j}) + \gamma h(p_{i,j})\tag{3}$$
$$\frac{\partial L(p_{i,j})}{\partial p_{i,j}}= -1 -\log p_{i,j} + \beta (i^2+j^2) + \gamma =0\tag{3}$$
$$ p_{i,j} = A e^{B(i^2+j^2)}=A e^{B i^2}e^{Bj^2} \tag{4}$$
Plugging $(4)$ into $(1)$ and $(2)$ we obtain $A,B$. Notice that the components are independent (which looks right). It remains to check if this critical point is truly a global maximum, and if it makes sense to use the restriction $ E (||X||^2) = \alpha$ instead of the original one.