I am now studying the channel coding theorem in information theory. The proof of the theorem in the textbook (by Cover & Thomas, Theorem 7.7.1) picks a fixed $p(x)$ for the codewords first, and then shows the probability of error goes to zero if $R<I(X;Y)$. Therefore the distribution $p^*(x)$ that maximizes the mutual information is chosen so that the condition $R<I(X;Y)$ can be replaced by $R<C$, where $C$ is the channel capacity. That is, $R<C$ implies probability of error tends to zero.
This proof seems to be a standard proof as I have seen in many materials.
Problem: However, at the same time this proof also assumes uniform distribution on the input messages. The problem is: if the input message $W$ is assumed to be uniformly distributed, then the codeword $X^n(W)$ should be also uniformly distributed. How can we choose the distribution of $X$ to be $p^\ast(x)$? Or, if the distribution of $X$ is $p^*(x)$, how can the distribution of $W$ be uniform distribution?
Is there something wrong in my understanding? How to undertand this theorem correctly?
I don't think so. For example, let $x$ be Bernoulli with $p=P(X=1)=1/3$. Take $R=1/2$ and $n=6$. Then we have $M=2^3=8$ messages (uniformly distributed), which are mapped to $8$ codewords. Each codeword is a binary string of length $n=6$, and (according to the random coding design) each will have in average $2$ ones and $4$ zeros.
Hence, in spite of all inputs (and codewords) being equiprobable, the bits in the coding ($x$) are not.