Say $U\leq V$ for a vector space $V$, and $v \space \in \space V$. We define an affine subspace to be a coset of a vector subspace, namely $$v+U=\{v+u | u \space\epsilon\space U\} \subset V$$
However it can be proven that this is only a subspace when $v \space\in\space U$.
So my question is, in the case when $v \not\in U$, why is the coset of the vector subspace still called an affine subspace when it is not actually a vector space?
The phrase "affine subspace" has to be read as a single term. It refers, as you said, to a coset of a subspace of a vector space. As is common in mathematics, this does not mean that an "affine subspace" is a "subspace" that happens to be "affine" - an "affine subspace" is usually not a subspace at all.
But there is a relationship between affine subspaces and (genuine) subspaces, which is the motivation for the terminology. Every subspace is an affine subspace, and every affine subspace can be translated to become a subspace. So, geometrically, subspaces and affine subspaces have a similar structure.
There are many other examples in mathematics where this kind of thing happens - where a modifier to a word creates a compound term which refers to something that the original word did not refer to. Another example is a "skew field", which is not usually a field.