Let $(Y, \mathcal{A}, \mu)$ be a probability space and $T:Y\rightarrow Y$ a measure preseving map. Define $U=\{f\in L^2(Y): f\circ T=e^{-i\theta}f~~\text{a.e.}\}$ where $\theta\not \in 2\pi\Bbb{Z}$. I want to show that if $T$ is strong mixing, then $U=\{0\}$.
Clearly $\{0\}\subset U$. So one pick $f\in U$, i.e. $f\circ T=e^{-i\theta}f$. Now we can use the following fact with $g=1$
$T$ strong mixing iff for all $f,g\in L^2(Y)$, $\int_Y f(T^n(y)) g(y)~d\mu=\int_Y f(y)~d\mu \int_Y g(y)~d\mu$
Then we deduce that $$\lim_{n\rightarrow \infty} \int_Y f(T^n(y))~d\mu=\int_Y f(y) ~d\mu$$ or equivalently $$\lim_{n\rightarrow \infty} (e^{-in\theta}-1)\int_Y f(y) ~d\mu=0$$Since $\theta\not \in 2\pi \Bbb{Z}$, we deduce that $e^{-in\theta}\neq 1$, therefore $\int_Y f(y) ~d\mu=0$. But from here I cannot deduce that $f=0$.
Can someone help me how to conclude? I also thought about working with $A:=\{y\in Y: f(y)\geq0\}$ and then define $g(y)=\Bbb{1}_A$ so that I somehow can guarantee that $f\geq 0$ and deduce $f=0$ but this also did not work.
$\newcommand{\d}{\,\mathrm{d}}$You did not actually use the mixing hypothesis in writing: $$\lim_{n\to\infty}\int_Yf\circ T^n\d\mu=\int_Yf\d\mu$$Since in fact, by measure-preservation, it is true that: $$\int_Yf\circ T^n\d\mu=\int_Y f\d\mu$$Is true for every $n\in\Bbb N_0$. In this way, you can argue $\Bbb E[f]=0$ without using the mixing hypothesis (only needing $e^{-i\theta}\neq1$).
Let me use this as my reference. You probably already know this, but for completeness I state the results I use:
The fundamental definition is this:
Theorems 9.4 and 9.6 combine to tell us several equivalent statements, in particular:
Let's fix $f\in U$ and set $p=2$ - then $q=2$. I'm going to keep the notation where the dynamic $T$ is written as $\varphi$ and the Koopman operator $T_\varphi$ is written as $T$, because $T_T$ looks silly.
Our goal is to show that $f=0$. In $L^2$, this is equivalent to say: $$\int_Yf^2\d\mu=0$$
For any $n\in\Bbb N_0$ it is true that: $$e^{-in\theta}\int_Yf^2\d\mu=\int_Y(T^nf)f\d\mu$$
Hence: $$\lim_{n\to\infty}e^{-in\theta}\|f\|_2^2=\Bbb E[f]^2=0$$
Since the complex modulus function is continuous, it is trivial that: $$\lim_{n\to\infty}\|f\|_2^2=0$$That is, $\|f\|_2=0$ and $f=0$ (up to the almost everywhere equivalence classes).
Using $g=f$ in point $(2)$ is justified only because $f$ is in $L^2$ and the Young conjugate of $2$ is $2$.
Remark:
It is also shown in the book, in chapter $7$, that the fixed space, that is, the eigenspace with eigenvalue $1$, of the Koopman operator is one dimensional when the system is ergodic - and in chapter $9$ it is shown every strongly mixing system is ergodic. Moreover in chapter $7$ it is shown that all eigenvalues of the Koopman operator live on the unit circle.
The original question was essentially asking: show that the only eigenvalue of $T$ that has unit norm is $1$, when considered in $L^2$. Coupled with the above, that means the only eigenvalue whatsoever of the Koopman operator $T:L^2\to L^2$ for a strongly mixing system is $1$, and that the eigenspace is one dimensional hence clearly spanned by the (a.e. equivalence classes of the) constant functions.
In your original notation, that means: $$U’:=\{f\in L^2(Y):\exists\lambda\in\Bbb C,\,f\circ T\equiv\lambda f\text{ a.e.}\}=\{z\cdot\mathbf{1}:z\in\Bbb C\}$$