I'm trying to learn signals and systems on my own and the book I'm using refers to the coefficients of the Fourier series as the "frequency domain."
So suppose we have a signal in time $x_1(t) = \sum_{k = -\infty}^{\infty}X[k]{e^{j\frac{2\pi kt}{T}}}$
Why are our X[k]'s called the frequency domain?
My guess: We have $\omega = 2\pi f = \frac{2\pi}{T}$ which is angular frequency right? So in our exponent $k$ is multiplying our angular frequency. Is that why?
Refer to this fantastic animation from Wikipedia (here we have interchanged $x_1(t)$ in your notation with $f(t)$):
Here you can see that a simple function $f$ is being decomposed into a sum of simple sinusoidal waves, some more important than others (as measured by their amplitude). (For the purposes of this answer, you can think of the simple sinusoidal waves of frequency $\omega$ as equivalent to waves of the form $e^{i\omega t}$. These waves are called plane waves, and intuitively can be thought of as a complex extension of the basic sinusoidal waves. But again, just assume they are equivalent for our purposes.) In your notation, $X[k]$ is the (potentially complex) amplitude of the wave with frequency $2\pi k / T$ in this decomposition. The bigger the magnitude of the $X[k]$, the more "important" it is in constituting the original function $f$.
The function $f(t)$ is referred to as the time domain expression of the signal, because we can input any given time and retrieve the value of the signal at that time. So why are the $X[k]$'s called the frequency domain? It's because you can think of $X$ as a function taking in a parameter $k$ corresponding to the frequency $2 \pi k / T$ and returning an amplitude whose magnitude denotes the contribution of the simple sinusoidal wave of that frequency to the signal. This is an equally valid way of encoding a signal, just with information about the frequencies of the simple sinusoidal (plane) waves that constitute it.