Let's take the matrix $A$: \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}
This transformation on $\mathbb{R}^2$ is a shear. Note that $G = \langle A^n \mid n \in \mathbb{Z} \rangle$, is a subgroup of : $GL_2(\mathbb{C})$. Moreover the subspace $E = \{\lambda \cdot {}^t (1 \quad0 ) \mid \lambda \in \mathbb{R}\}$ is an invariant subspace of $\mathbb{R}^2$. Yet by Maschke theorem, there exist a supplementary of $E$ that is an invariant subspace of $\mathbb{R}^2$, but when I am trying to visualize the transformation associated with $A$ I don't find this supplementary.
I feel like only the $x-$axis is invariant by a shear. That's why I think I am wrong somewhere. Thank you for your help.
You have just proved that Maschke's theorem need not hold for infinite groups. Your $G$ is isomorphic to $\mathbb{Z}$, and Maschke's theorem does not hold for $\mathbb{Z}$. Have another look at the proof of Maschke to see where the finiteness of $G$ is used.