A basis is a set of linearly independent vectors that span a vector space $V$. A coordinate vector $v$ can be used to reference vectors within this space. The components of v are the coefficients of a linear combination of the basis vectors. Sticking with subspaces of $\mathbb R^2$ (for simplicity), any basis vector we choose can be expressed as a linear combination of the vectors comprising the standard basis.
For example, consider a basis $\left\{ a,b\right\}$ where: $$a=\begin{bmatrix} 2 \\ 0 \end{bmatrix} \, \,\,\,b=\begin{bmatrix} 0 \\ 2 \end{bmatrix}$$ The vectors of this basis can be rewritten as: $$a = 2i + 0j \,\,\,\,\,\,\,\,b = 0i + 2j$$ $a$ and $b$ are just coordinate vectors. So what are i and j coordinates of? Themselves? The vector $a$ in our basis could be rewritten as the following:
$$a = 2i + 0j = 2\begin{bmatrix} 1 \\ 0 \end{bmatrix} + 0 \begin{bmatrix} 0 \\ 1 \end{bmatrix} = 2 * 1 * i + 0 * 0 * j + 0 * 0 * i + 0 * 1 * j = \,\,... $$
This interpretation of the standard basis vectors seems circular and recurses infinitely. Is there a better way to think about these entities?
When you have a basis of $V = \mathbb{R}^2$, each vector in $V$ can be written as a linear combination of the basis vectors.
When you use the standard basis $i = (1,0)$, $j=(0,1)$ each vector $(a,b)$ is its own "coordinate vector" since $$ (x,y) = xi + yj. $$
If you think of another basis, say $v = (1,2), w = (1,3)$, then you can express any vector as a linear combination of $v$ and $w$. So for example $$ 2v -3w = (-1, -5) $$ so the coordinate vector of $(-1, -5)$ with respect to the basis $\{v,w\}$ is $(2, -3)$
Note: In basis $\{v,w\}$ the coordinate vector of $v$ is $(1,0)$ because $$ v = 1v + 0w. $$ but you should not think of that as the same object as the standard basis vector $i$.
That is why I think the phrase "coordinate vector" is ill chosen, and may be part of what is confusing you. But if that's what your book and course use, learn it.