Let $ G $ be a connected Lie group and $ \Gamma $ a subgroup. Let $ \mathfrak{g} $ be the Lie algebra of $ G $ and $ ad: G \to GL(\mathfrak{g}) $ be the adjoint representation of $ G $ and let the $ ad: \Gamma \to GL(\mathfrak{g}) $ be the restriction of the adjoint representation to $ \Gamma $.
We say a (proper closed) subgroup is maximal if it is maximal among all proper closed subgroups of $ G $.
Suppose that $ \Gamma $ is a discrete subgroup of a connected Lie group $ G $ and moreover that $ \Gamma $ is self-normalizing and irreducible in the adjoint representation. Does that imply that $\Gamma $ is a maximal subgroup of $ G $?
Note that the normalizer of a closed subgroup is always closed. Thus a maximal subgroup is always either normal or self-normalizing. But a (proper closed) normal subgroup $ H $ of a connected Lie group $ G $ is never maximal since that would imply $ G/H $ has no closed subgroups, which is impossible for a connected Lie group.
Here is some motivation:
Consider the compact group $ G=\operatorname{SO}_3(\mathbb{R}) $. The closed subgroups of $ G $ (other than the trivial group 1 and the whole group $ G $) are exactly $ \operatorname{O}_2$, $\operatorname{SO}_2 $ and the finite groups $ C_n$, $D_{2n}$, $T \cong A_4$, $O \cong S_4$, $I \cong A_5 $ (cyclic groups with $ n $ elements, dihedral groups with $ 2n $ elements and the three symmetry groups of the platonic solids). The normalizers of these groups are as follows:
\begin{align*} G&=N_G(G)=N_G(1) \\ \operatorname{O}_2&=N_G(\operatorname{O}_2)=N_G(\operatorname{SO}_2)=N_G(C_n) \\ I&= N_G(I) \\ O&=N_G(O)=N_G(T)=N_G(D_4) \\ D_{4n} &= N_G(D_{2n}) \end{align*} where in the last equation $ n \geq 3 $.
Observe that in the example above the maximal subgroups exactly coincide with the self-normalizing subgroups. Namely, $$ \operatorname{O}_2, I,O. $$ Also note that of the three maximal subgroups it is exactly the the two discrete groups $ I,O $ that are irreducible in the adjoint representation (which here is just the standard 3d irrep of $ SO_3(\mathbb{R}) $).
That the maximal subgroups are all self-normalizing is not too surprising. The normalizer of a closed subgroup is always closed. Thus a maximal subgroup is always either normal or self-normalizing. But a (proper closed) normal subgroup $ H $ of a connected Lie group $ G $ is never maximal since that would imply $ G/H $ has no closed subgroups, which is impossible for a connected Lie group.
No.
The complex reflection group with Shephard-Todd number 29 (I will call it ST-29) is a finite subgroup of $ SU_4 $ of order 7680. It is a unitary 2-design, see for example
https://arxiv.org/abs/1810.02507
part A3 of theorem 7. A subgroup of $ SU_n $ is a unitary 2 design if and only if it is Ad-irreducible. Thus ST-29 is Ad-irreducible. To see that ST-29 is self-normalizing, consider the following. ST-29 is exactly group XXVIII from
https://arxiv.org/abs/1810.02507
The normalizer of ST-29 must be finite since any infinite closed subgroup of $ SU_n $ containing a 2 design is all of $ SU_n $. But ST-29 is primitive so any finite subgroup containing it is a primitive finite subgroup. There are only 30 such subgroups and they are all explicitly listed in the 2nd reference. Of the 30 the only ones with order divisible by 7680 are ST-29 itself and then the groups labelled XXIX and XXX. One can check, using GAP for example, that XXIX and XXX have no normal subgroups of order 7680. Thus ST-29 is self normalizing and Ad-irreducible.
However ST-29 is contained in group XXX. So ST-29 is a self normalizing Ad-irreducible group which is not maximal.