From wikipedia https://en.wikipedia.org/wiki/Tensor_algebra
"It is the free algebra on $V$"
I understand that as "It's the free algebra on the alphabet of $\{ v_i: i \in I\} \cup \{ \emptyset \}$", where $\{ v_i: i \in I\}$ is some basis of $V$.
I think I undestand the definition of a tensor algebra through tensor products properly, and I think I can see that a free algebra on the alphabet above gives the same thing, but I want to make sure I get this right and avoid some future misunderstanding - maybe "free algebra on $V$" could actually mean "free algebra on alphabet $\{v : v \in V \} \cup \{ \emptyset \}$" - though I don't understand how that would make sense.
Do I understand this correctly?
It looks like you're having some confusion about what "free" is meaning here.
I can see you'd like to express everything in terms of a basis $\{v_1,\ldots,v_n\}$ (and for some reason I can't fathom, you included $\emptyset$). Yes, you can fix a basis for $V$ and then make a basis of $V^k$ and then a basis of $\oplus_{i=0}^\infty V^k$, but that isn't really an efficient way to go about it. It seems like that would get a little confusing when bookkeeping the extrapolated basis.
The sense in which it is "free" is the one described at the wiki page: it's left adjoint to the forgetful functor from algebras to vector spaces.
It enjoys a universal property like this: if $\phi:V\to A$ is any $k$ linear transformation into a $k$ algebra $A$, you have a uniquely determined $k$ algebra homomorphism $T(V)\to A$ extending it. It is free in the sense of being "the most general algebra generated by $V$."