Let $N\subset \mathbb N$ be a finite set. Define the sequence of functions $f_{k}: N\to\mathbb R$. Let $X_k$ be a homogeneous irreducible aperiodic Markov chain taking values in $N$ with no absorbing states. Assume that $f_{k+1}(X_k)$ satisfies the following: \begin{align}\tag{1} | f_{k+1}(X_k) |\leq \delta \Vert f_k\Vert_\infty \ \ \ \text{a. s.} \end{align} with $\delta\in(0,1)$.
Question: Can we know something about the convergence of $f_k$? If nothing, what kind of conditions do we need to get some kind of convergence?
Attempts. I think that $f_k\to 0$ at least pointwise. But I cannot use usual tools, because there is some randomness going on here. My stochastic process knowledge is weak, but I tried the following. Define $\tau_n:=\sum_{j=1}^n Y_j$ where $Y_j$ is the time it takes to get to see each state of $X_k$ at least once starting from where we were last in $Y_{j-1}$. Then from $(1)$ we immediately see the following:
$$\Vert f_{\tau_{n+1}}\Vert_\infty \leq \delta \Vert f_{\tau_n}\Vert_\infty$$ Clearly $f_{\tau_n}\to 0$ as $n\to\infty$ (I don't know what kind of convergence do we have here?). It seems that now I have something like: given $\varepsilon>0$, then there is some random time $\tau$ such that for all $k>\tau$ one has:
\begin{align}
\Vert f_k \Vert_\infty \leq \Vert f_{\tau}\Vert_\infty <\varepsilon
\end{align}
Can we conclude $f_k\to 0$ then? What bothers me is the random time $\tau$. Also I think that I need some assumptions on $\tau_n$ like $\tau_n<\infty$ a.s., is that true?
I appreciate any help. Even hints like "You need to study that part of that book.". Thanks in advance!
(This is actually part of a larger problem where I'm trying to show that the error made in an algorithm will be zero eventuallly)
For simplicity, assume that the initial condition of the Markov chain is fixed, say, $X_0=0$ surely, where $0$ is a particular state in $N$. Your equality (1) is equivalent to: For all $k \in \{0, 1, 2, ...\}$ we have $$ |f_{k+1}(X_k)|\leq \delta \max_{x \in N} |f_k(x)| \quad \mbox{(with prob 1)} \quad (1)$$ where $\delta \in (0,1)$ is a given constant.
If the chain is irreducible and aperiodic
Since the chain is finite state, irreducible, and aperiodic, there is a (deterministic) time index $K$ such that for every $y \in N$, we have $$ P[X_k = y] >0 \quad \forall k \geq K$$ (This can be proven from the fact that aperiodic means there is a $\tilde{K}$ such that it is possible to go from state $0$ to state $0$ in $k$ hops whenever $k \geq \tilde{K}$.) Fix $y \in N, k \geq K$. If $f_{k+1}(y)>\delta \max_{x \in N} |f_k(x)|$ then $$P\left[|f_{k+1}(X_k)|> \delta \max_{x \in N}|f_k(x)|\right] \geq P[X_k=y]>0$$ which contradicts (1). Thus, $$ |f_{k+1}(y) |\leq \delta \max_{x \in N} |f_k(x)| \quad \forall y \in N, \forall k \geq K$$ Taking a max of both sides over all $y \in N$ gives $$\max_{y \in N} |f_{k+1}(y)|\leq \delta \max_{x\in N} |f_k(x)| \quad \forall k \geq K$$ and so $\lim_{k\rightarrow\infty}\max_{x \in N} |f_k(x)|= 0$.
If the chain is irreducible but not aperiodic
Here is a counter-example: Let $N=\{0,1\}$ and consider the Markov chain that deterministically alternates between states 0 and 1 over time $k \in \{0, 1, 2, ...\}$: $$ X_k = \left\{ \begin{array}{ll} 0 &\mbox{ if $k$ is even} \\ 1 & \mbox{ if $k$ is odd} \end{array} \right.$$ It is easy to construct functions $f_k:\{0,1\}\rightarrow\mathbb{R}$ that satisfy (1) with $\delta = 1/2$ such that $$ \lim_{k\rightarrow\infty}\max_{x \in \{0,1\}} |f_k(x)| = \infty$$