Let $\{X_{t}\}$ be a stationary sequence and let $\mathbf{X}_{n} := \left(X_n,\cdots,X_{n}\right)^{T}$ denote the vector of the first $n$ observations from this sequence. We denote the autocovariance function (ACF) by $\gamma(h) := \text{COV}(X_{t},X_{t+h})$. Now one can show the following:
Lemma: If $\gamma(0)>0$ and $\gamma(h) \rightarrow 0$ for $h \rightarrow \infty$ then the covariance matrix $\Gamma_{n} = \text{COV}(\mathbf{X}_{n}) = \left[\gamma(i-j)\right]_{i,j = 1\cdots n}$ is nonsingular for every $n$.
I´m looking for a counter example for the "$\Leftarrow$" direction. How could a process $X_{t}$ look like for which $\Gamma_{n}$ is non-singular, but $\gamma \not\rightarrow 0$ as $h \rightarrow 0$? I was experimenting already with something like $X_{t} = Z_{1}\text{cos}(ct) + Z_{2}\text{sin}(ct)$ where $Z$~$\text{WN}(0,1)$. Then I get $\gamma(h) = \text{cos}(ch)$, which obviously does not converge as $h \rightarrow \infty$, but unfortunately $\Gamma_{n}$ ist singular for $n=3$ then too. Has anyone a better idea? I would really appreciate any comments/answers/help with this! Thank you in advance!