Let $A$ and $B$ be $n\times n$ matrices, and let $[A,B]=AB-BA$ be the commutator of $A$ and $B$.
Then, if we consider the matrix exponential $e^B=\sum_n \frac{B^n}{n!} $, we have $$ [A,e^B]=\sum_n \frac{1}{n!}[A,B^n] $$
Now, if $[A,B]=0$, then $[A,B^n]=0$ $\forall n$ and therefore $[A,e^B]=0$. But what about the converse? Is it true as well?
In other words, does $[A,e^B]=0$ imply $[A,B]=0$ ?
No. E.g. let $B=\pmatrix{0&0\\ 0&2\pi i}$. Then $e^B=I$. Now, all matrices commute with $e^B$, but some of them don't commute with $B$.
More generally, if $B$ is diagonalisable over $\mathbb C$ and it possesses two eigenvalues $\lambda_1$ and $\lambda_2$ such that $\lambda_1-\lambda_2$ is a nonzero integer multiple of $2\pi i$ (i.e. $B$ is a diagonalisable matrix whose eigenvalues are not $2\pi i$ congruence-free), then the family of matrices that commute with $B$ is a proper subset of the family of matrices that commute with $e^B$. Hence this $B$ serves as a counterexample to your hypothesis. In particular, any such counterexample must be non-Hermitian.