I was thinking about stochastic processes and when they are stationary.
First of all, if I consider a fair coin and I look at $X_n$, the outcome of the $n$-th coin toss, I get in all the process $(X_n)$. Is this process stationary?
Then I want to extend my experiment:
Step : Choose a coin - there are two, a fair one (probability 1/2) and an unfair one (probabilities p and 1-p).
Step: Toss the choosen coin $n$ times.
In this case, is $(X_n)$ stationary? How can I compute the entropy rate?
The concept of stationary stochastic processes, informally, says that its statistics don't depend on time. This implies that the marginal probabilities don't depend on time $P(X_n) = P(X_m)$, but also the joint probabilities, say $P(X_n,X_{n-k})=P(X_m,X_{m-k})$, and not only the pairs but the triplets and so on. This looks quite difficult to test, but in practice it's often not difficult.
In particular, if the process is "white" (values $X_n,X_m$ are independent for any $n \ne m$), then the process is stationary iff the marginal probabilities are constant.
In your first case, you see that the values are independent, and the marginal probabilities don't depend on $n$: $$P(X_n=0) = 1/2\\P(X_n=1) = 1/2$$
Hence, yes, the process is stationary (this also happens if the coin is unfair, as long as it's the same coin).
The second one is a little trickier. The problem here is that the succesive values of $X_n$ are not independent: knowing the previous one gives us some information about which coin are we using.
In these cases, a natural trick is to add a random variable $Y$ which correspond to the result of the first step. Conditioning on the value of this variable you can show that the process is indeed stationary (as intuition should tell you), and also allows to compute the entropy rate. I hope you can go on from here.
Update: For the second:
$$P(X_i,X_j)= \sum_Y P(X_i X_j \mid Y ) P(Y) =\sum_Y P(X_i \mid Y )P(X_i \mid Y ) P(Y) \tag{1} $$
That is, $X_n$ conditioned on $Y$ is iid. Then $X_n$ is a mixture iid (hence stationary) process. This does not imply that $X_n$ is iid (it isn't), but it implies that $X_n$ is stationary, because the joint probabilities (as $(1)$, and all the others) don't depend on the time index.