So I've been reading up on Lie Groups and Lie Algebras and the Baker-Campbell-Hausdorff formula. I understand how the formula works and that most of the time the nested commutators vanish at a certain point, but I'm wondering if there exist pairs of matrices whose nested commutators never vanish and what they'd look like.
I can't find anything on the subject and it's driving me nuts.
Sure: In $\mathfrak{sl}(2)$, the usual basis elements $e, f, h$ satisfy \begin{align*} [e, f] &= h & [h, e] &= 2e & [h, f] &= 2f. \end{align*} Thus the $k$-fold nested commutator $[h, [h, \dots, [h, f] \cdots ]] = 2^k f$ is nonzero for all $k > 0$. More generally, the condition that every $k$-fold nested commutator vanishes for some fixed $k$ (i.e., not depending on the elements in that commutator) is called nilpotency.