Interpreting $h(T, \zeta) = \displaystyle{\lim_{n\to \infty}}H(\zeta | T^{-1}\zeta^n)$ from Brin and Stuck

Question

Let $T:(X, B, \mu) \rightarrow (X, B, \mu)$ be a measure preserving system. For a finite partition $\zeta$ of $X$ we define

$\zeta^n:=\zeta \vee T^{-1}\;\zeta \; \vee ... \vee \; T^{-(n-1)}\zeta$

Then the measure theoretic entropy of $T$ with respect to $\zeta$ is defined as

$h(T, \zeta ):=\displaystyle{\lim_{n\to \infty}}\frac{1}{n}H(\zeta^n)$, where $H(\eta)$ denotes the entropy of the partition $\eta$.

Brin and Stuck prove that

$h(T, \zeta) = \displaystyle{\lim_{n\to\infty}} H(\zeta | T^{-1}\zeta^n)$, where $H(\eta | \epsilon)$ is the conditional entropy of $\eta$ with respect to $\epsilon$.

They remark that this proposition shows that "$h(T,\zeta)$ is the average information added by the present state on the condition that all past states are known."

I'm unsure of how $T^{-1}\zeta^n$ can be interpreted as the past in the limit. Can anyone provide an explanation?

Answer 1

I would erase the word "added", not really appropriate there.

You should think as follows (although this is hard to make rigorous, if at all possible):

The present "states", in case they come from something at $-\infty$, come from the $\sigma$-algebra generated by all partitions $T^{-1}\zeta^n$ (the latter includes the past "states" provided that $n$ is arbitrary). So, in view of the formula that they prove, the entropy with respect to a partition $\zeta$ is positive if and only if some element of $\zeta$ is not is that $\sigma$-algebra (if you think of the way in which conditional entropy is defined this is really what needs to happens so that it can be positive). This leads us to the interpretation given in the book, of course without the word "added" since we add nothing at all, we simply compute something.

To be precise, they should really have written something like: "is the limit of the average information at the present state given that a growing number of past states are known". This latter formulation avoids a reference to $\sigma$-algebras (although it is really unavoidable to refer to them with the formulation in the book).