How to solve $\lim_{x\to \pm \infty} (1+\frac 1x)^x$

Question

I have this problem: $\lim_{x\to \pm \infty} (1+\frac 1x)^x$

I understand everything except how to handle the $\pm \infty$. When I tried to solve it I reasoned like this, since both $1^\infty$ and $1^{-\infty}$ should be equal to one then the answer to the original question should be $1$. But the correct answer is $e$. I can solve problems like these when it's just $+\infty$ but how do I solve it when there is plus minus infinity? Any help is appreaciated.