Given a Hilbert space $\mathcal{H}$.
Consider an operator: $$T:\mathcal{D}(T)\to\mathcal{H}:\quad\overline{\mathcal{D}(T)}=\mathcal{H}$$
Regard a subspace: $$\mathcal{S}\leq\mathcal{H}:\quad\mathcal{H}=\mathcal{S}\oplus\mathcal{S}^\perp$$
Can it happen that: $$T\mathcal{S}\subseteq\mathcal{S}\quad T\mathcal{S}^\perp\nsubseteq\mathcal{S}^\perp$$ Does someone have an example at hand?
Nilpotents
Consider the nilpotent matrix: $$N:=\begin{pmatrix}0&1\\0&0\end{pmatrix}$$ Regard the closed subspace: $$\mathcal{S}:=\{\begin{pmatrix}a\\0\end{pmatrix}:a\in\mathbb{C}\}$$ Then it fails to reduce: $$N\mathcal{S}\subseteq\mathcal{S}\quad N\mathcal{S}^\perp\nsubseteq\mathcal{S}^\perp$$
Ladders
Consider the annihilation operators: $$a(\eta):\mathcal{F}(\mathcal{h})\to\mathcal{F}(\mathcal{h}):\quad a(\eta)\bigotimes_{i=1}^k\sigma_i:=\langle\eta,\sigma_n\rangle\bigotimes_{i=1}^{k-1}\sigma_i$$ Regard the bosonic resp. fermionic Fock space: $$\mathcal{F}^\pm_0(\mathcal{h}):=P^\pm\mathcal{F}_0(\mathcal{h})$$ Though invariant they fail to be reducing as: $$P^\pm a(\eta)P^\pm=a(\eta)P^\pm\quad a(\eta)P^\pm\neq P^\pm a(\eta)$$ (Note that for creation operators they're not even invariant.)
Hamiltonians
For Hamiltonians: $$H=H^*:\quad H\mathcal{S}\subseteq\mathcal{S}\implies H\mathcal{S}^\perp\subseteq\mathcal{S}^\perp$$
That follows from: $$\psi\in\mathcal{S}^\perp\cap\mathcal{D}:\quad\langle H\psi,\chi\rangle=\langle\psi,H\chi\rangle=0\quad(\chi\in\mathcal{S}\cap\mathcal{D})$$
And assertion follows by density.