Consider two irreducibles of a topological group $G$, acting in respective Hilbert spaces $\,\mathbb V\,$ and $\,{\mathbb{V}}^{\,\prime}\,$.
Schur's lemma says:
An intertwiner $\,M\,:\; {\mathbb{V}}\longrightarrow{\mathbb{V}}^{\,\prime}\,$ of two irreducibles of a group is either zero or isomorphism.
To prove it, we first show that both $\,{\operatorname{img}} \,M\,$ and $\,{\ker} \,M\,$ are invariant subspaces. Then two observations are made.
Observation 1. For the invariance of $\,{\operatorname{img}} \,M\,$ to agree with the irreducibility, $\,{\operatorname{img}} \,M\,$ must coincide either with the space $\,{\mathbb{V}}^{\,\prime}\,$ or with its zero vector $\,\vec{0}^{\,\prime}\,$.
Observation 2. For the invariance of $\,{\ker} \,M\,$ to agree with the irreducibility, $\,{\ker} \,M\,$ must coincide either with the space $\mathbb V$ or with its zero vector $\,\vec{0}\,$.
Summing up these observations, we conclude that $M$ is either zero or bijective and therefore invertible.
Now, my question. Is it really true that $\,{\operatorname{img}} \,M\,$ itself must coincide either with $\,{\mathbb{V}}^{\,\prime}\,$ or $\,\vec{0}^{\,\prime}\,$? $\;$Or is it sooner the closure of $\,{\operatorname{img}} \,M\,$ that must satisfy this requirement? I am enquiring, because a representation is always defined in a Hilbert space or in a closed subspace thereof.
The same question about $\,{\ker} \,M\,$. Should we prove that it is actually its closure that is invariant?
If it is the closures of $\,{\operatorname{img}}\, M\,$ and $\,{\ker}\, M\,$ whose invariance needs to be proven, do we need to impose additional requirements on the representations and/or on the intertwiner $\,M\,$? (say, boundedness?)
In what follows, I will summarize the main points from the discussion in the comments to the OP above, and then proceed with a qualified answer.
There are two different notions of irreducibility for a representation $U$ of a group $G$ in a Hilbert space $\mathscr{H}$:
Recall that a subspace of a Hilbert space is complete (hence, a Hilbert space itself when endowed with the scalar product of the ambient space) if and only if it is closed. Obviously, algebraic irreducibility of $U$ only depends on the vector space structure of $\mathscr{H}$ (the scalar product and the topology induced by it are irrelevant) and entails topological irreducibility, whereas the converse is not always true.
If $U$ is algebraically irreducible, then Schur's lemma is just a statement on linear algebra and is proven exactly as you wrote above - let us review the argument in a suitable form for later use. If $U_i$ are representations of $G$ in respective Hilbert spaces $\mathscr{H}_i$, $i=1,2$ and $T:\mathscr{H}_1\rightarrow\mathscr{H}_2$ is a linear map such that $TU_1(g)=U_2(g)T$ for all $g\in G$ (i.e. $T$ is an intertwiner of $U_1$ with $U_2$), then $U_1(G)\ker(T)\subset\ker(T)$ and $U_2(G)\mathrm{ran}(T)\subset\mathrm{ran}(T)$, where $\ker(T)=T^{-1}(0)=T^{-1}(\{0\})\subset\mathscr{H}_1$ is the kernel of $T$ and $\mathrm{ran}(T)=T(\mathscr{H}_1)\subset\mathscr{H}_2$ is the image (range) of $T$, hence $\ker(T)$ and $\mathrm{ran}(T)$ are $U(G)$-invariant subspaces. If $U_1,U_2$ are (algebraically) irreducible, this entails that $\ker(T)=\{0\}$ or $\mathscr{H}_1$ and $\mathrm{ran}(T)=\{0\}$ or $\mathscr{H}_2$, noting that $\ker(T)=\mathscr{H}_1$ and $\mathscr{ran}(T)=\{0\}$ are equivalent statements. This entails that either:
In the case that $U_1,U_2$ are topologically irreducible representations of $G$, one needs further hypotheses to proceed, unless $\mathscr{H}_1,\mathscr{H}_2$ are finite dimensional (in which case both notions of irreducibility are equivalent, for then any subspace is closed). For topologically irreducible representations, it is customary to require the following:
A few comments about assumptions 1.-4. are in order. For the Schur lemma to hold, one does not really need 1. or 3., but one does need 4. and something stronger than 2. - more precisely, we need that $U_i(g)$ is unitary for all $g\in G$, $i=1,2$ (the reason why will become clear shortly). If $G$ is Hausdorff and compact one can get away with 1.-4. without requiring unitarity of $U_i$ a priori, for the Haar probability measure on $G$ then allows one to obtain from the scalar product on $\mathscr{H}_i$ a new scalar product w.r.t. which $U_i(g)$ is unitary for all $g\in G$ (Weyl's unitary trick). Now we are ready to state the appropriate topological version of Schur's lemma:
Notice that the boundedness of $T$ is used to ensure not only that $T$ is everywhere defined but also that the intertwining relation makes sense everywhere. Moreover, in this case $\ker(T)$ is necessarily closed but $\mathrm{ran}(T)$ not necessarily so if $\mathscr{H}_1$ and $\mathscr{H}_2$ are infinite dimensional, so the proof of the Lemma (which is essentially the same as the one provided in MO by Johannes Ebert) will necessarily be more involved. For the sake of readability for the math.SE audience, I am being a bit verbose here with (most of) the details since the MO answer is quite short.
Proof of Lemma: If $TU_1(g)=U_2(g)T$, then taking adjoints yields $$U_1(g)^*T^*=U_1(g^{-1})T^*=T^*U_2(g)^*=T^*U_2(g^{-1})\ ,$$ where the first and last identities follow from the fact that $U_1,U_2$ are unitary. This implies that if $T$ is a bounded intertwiner of $U_1$ with $U_2$, then $T^*$ is an intertwiner of $U_2$ with $U_1$ (in that order). Hence we conclude that $$T^*TU_1(g)=T^*U_2(g)T=U_1(g)T^*T$$ for all $g\in G$, that is, $T^*T$ is a bounded intertwiner of $U_1$ with itself, that is, $T^*T$ commutes with $U_1(g)$ for all $g\in G$. Since $T^*T$ is self-adjoint, we can apply the spectral theorem: the spectrum $$\sigma(T^*T)=\{\lambda\in\mathbb{C}\mid T^*T-\lambda\mathbf{1}\text{ is not invertible}\}$$ of $T^*T$ is a nonvoid closed (and bounded, since $T^*T$ is bounded) subset of $\mathbb{R}$ (even if $T^*T$ has no eigenvalues!), and there is a Borel measure $P$ on $\mathbb{R}$ supported in $\sigma(T^*T)$ taking values in the orthogonal projections on $\mathscr{H}_1$ such that $T^*T$ admits the spectral integral representation $$T^*T=\int_{\sigma(T^*T)}\lambda dP(\lambda)\ .$$ It turns out that $U_1(g)$ must then commute with the spectral projection $P(A)$ of $T^*T$ for every Borel subset $A\subset\mathbb{R}$ and for all $g\in G$.
To see this, recall how spectral projections are obtained - firstly, it is clear that any polynomial in $T^*T$ commutes with $U_1(g)$. By the Stone-Weierstrass theorem applied to any closed interval containing $\sigma(T^*T)$, one sees that any continuous function of $T^*T$ commutes with $U_1(g)$. Finally, for every open interval $A$ of $\mathbb{R}$ we have by the monotone and dominated convergence theorems, and the fact that the characteristic function of $A$ is the pointwise limit of an increasing sequence of continuous functions, that $U_1(g)$ commutes with $P(A)$. This extends to any other Borel subset of $\mathbb{R}$ by approximation.
Notice now that every orthogonal projection $P$ is bounded, hence continuous and therefore with a closed kernel, and also has a closed range (for $\mathrm{ran}(P)=\ker(\mathbf{1}-P)$) - hence $\ker(P(A))$ and $\mathrm{ran}(P(A))$ are closed $U(G)$-invariant subspaces for every Borel subset $A\subset\mathbb{R}$. Like in the proof of the original (algebraic) version of Schur's lemma, this entails that $P(A)$ ie either zero or an isomorphism - in the latter case we must have $P(A)=\mathbf{1}=P(\sigma(T^*T))$, for this is the only nonzero orthogonal projection with that property. Since this holds for every Borel subset $A\subset\mathbb{R}$, $T^*T$ must be a (possibly zero) scalar multiple of the identity in $\mathscr{H}_1$. Likewise, we conclude that $TT^*$ is a bounded self-adjoint intertwiner of $U_2$ with itself and, by the same argument as above, $TT^*$ must be a (possibly zero) scalar multiple of the identity in $\mathscr{H}_2$. Since $$\langle\psi,T^*T\psi\rangle=\|T\psi\|^2\geq 0\ ,\, \langle\phi,TT^*\phi\rangle=\|T^*\phi\|^2\geq 0\ ,\quad\psi\in\mathscr{H}_1\ ,\,\phi\in\mathscr{H}_2\ ,$$ we must have $T^*T=\lambda\mathbf{1}$ and $TT^*=\mu\mathbf{1}$ for some $\lambda,\mu\geq 0$. The same identities above imply that if either $\lambda=0$ or $\mu=0$ then $T=0$, so let us assume that $\lambda,\mu>0$. In that case, $T$ must be bijective, for then $\ker(T)\subset\ker(T^*T)=\{0\}$ and $\mathrm{ran}(T)\supset\mathrm{ran}(TT^*)=\mathscr{H}_2$. By the open mapping theorem, $T^{-1}$ must then be continuous as well. $\Box$
The spectral theorem is precisely what allows us to circumvent the fact that $T$ may not have closed range if $\mathscr{H}_1$ and $\mathscr{H}_2$ are infinite dimensional (of course, it turns out that $T$ does have closed range if $U_1$ and $U_2$ are topologically irreducible, but we do not know that a priori). The proof of the Lemma also implies that, given a (bounded) intertwiner $T\in\mathfrak{B}(\mathscr{H})$ of a topologically irreducible representation $U$ of $G$ by bounded linear operators on a Hilbert space $\mathscr{H}$ with itself, we have that if $T$ has a spectral integral representation then $T$ must be a (possibly zero) scalar multiple of the identity (here unitarity of $U$ is not needed, for we do not need to deal with $T^*$ in this case). This is the case if:
I conclude with a small aside: the counterexample you displayed in your own MO answer does not really apply to Hilbert space representations of $G$, for the vector space $\mathbb{C}(x)$ of all rational functions of one variable $x$ with complex coefficients is not a Hilbert space. More precisely, no scalar product is provided, let alone one that makes $\mathbb{C}(x)$ complete.