This question is motivated by the following: Suppose we have a conditional distribution $p(Y|X)$, denoting a “channel” (in the information theory sense). Suppose we fix a particular sequence $x_1,...$, and think of it as some realization of a random sequence $X_1,...$. Then this induces a sequence of random variables $Y_1,...$, such that $Y_i$ is distributed independently by $p(Y|X=x_i)$. Note that this gives a different distribution for each $Y_i$, but nevertheless they are independent. Intuitively, $Y_1,...$ denotes a random output sequence, based on a fixed particular input sequence $x_1,...$ and where each output variable depends only on its particular input variable.
My sense is that even though the sequence is not iid, and therefore not stationary and not ergodic, there should still be a version of the law of large numbers that applies, so long as we make some not-very-restrictive assumptions on $p(Y|X)$ (maybe we need the domains to be finite?).
Is there a result analogous to the LLN for such a sequence?
The question is open-ended, but one resolution is via the notion of a conditional typical set, which is a pretty standard concept. See, for example, Section 2.3 of these notes (which turned up as one of the top things with a google search, but the concept is standard):
https://web.stanford.edu/class/ee376a/files/2017-18/lecture_15.pdf
This is made more precise with the method of types, etc. Assuming the alphabets of X,Y are finite.