Recently, when fiddling around with integration by parts, I noticed that it is possible to define infinite series that led to an integral. My calculus teacher noticed this, and told me to find
$$ \int \frac{e^x}{x}dx $$
which I already knew didn't have a elementary function definition. After integrating by parts a few times, I found that this was leading to the summation
$$ e^x\sum_{k=1}^{\infty}\frac{(k-1)!}{x^k} + C $$
or, rather
$$ \frac{e^x}{x}\sum_{k=0}^{\infty}\frac{k!}{x^{k}} + C $$
Which, to me, looked very similar to the Taylor series definition of e^x:
$$ \sum_{k=0}^{\infty}\frac{x^k}{k!} $$
In that
$$ \frac{k!}{x^{k}}^{-1} = \frac{x^k}{k!} $$
Is there some form of between what I have found and the exponential function's Taylor series that I don't yet understand?
You have just stumbled upon the Exponential integral (Ei) function, which is a non-elementary function. Without getting into too much detail, that means that the function cannot be fully simplified, and the best we can do is express it as an series, as you did.
I believe however that the following expansion is more widely used, due to simplicity: $$ \int \frac{e^x}{x}dx=\int\frac{1}{x}\sum_{k=0}^{\infty}\frac{x^k}{k!}=\sum_{k=0}^{\infty}\int\frac{x^{k-1}}{k!}=\sum_{k=0}^{\infty}\frac{x^{k}}{k!\cdot k} $$