In MIT Course No. $18.712$, Associative Algebra $A$ is defined as a vector space over a field $K$ with a bilinear associative map $A \times A \to A$, $(a,b) \to ab$. Then some examples are given, where I found a free algebra $A=k<x_1,x_2,...,x_n>$, basis for this algebra consists of words in letters $x_1,x_2,....,x_n$ with multiplication as concatenation of words.
My question is as $A$ is a vector space then what is the addition in $(A,+)$ abelian group? Can anyone help to prove that $A$ is vector space to show it is an associative algebra.
Here you have a set $M=\{\text{words in } x_1, \cdots, x_n\}$, together with concatenation as a map $M\times M\to M$. This turns $M$ into a monoid. Sometimes people denote the generators as a set $X=\{x_1, \cdots, x_n\}$ and this monoid (which is the free monoid on $X$) is denoted as $M:=X^*$.
Then the free algebra you described (the free associative unital $k$-algebra) as a vector space is $$ A:=k\langle X\rangle:=\bigoplus_{w\in X^*} kw $$ In other words, the words $w\in X^*$ are the basis for the underlying vector space of $A$ and a vector is nothing but $\sum_{w\in X^*}^\text{finite} a_ww$ with $a_w\in k$ ($w$ is a purely formal, if it confuses you replace it with $e_w$, which should mean that basis vector corresponding to $w$). The addition and scalar multiplication are now defined in the obvious way: $\sum a_w w+\sum b_w w=\sum (a_w+b_w)w$ and $\lambda\sum a_w w=\sum (\lambda a_w)w$.
Finally the associative unital algebra structure comes from maps $\mu: A\times A\to A$ which is defined as $\mu(\sum_w a_w w, \sum_{v} b_{v}v)=\sum_{wv} (a_wb_v) (wv)$, where the multiplication $a_wb_v$ is in $k$ and $wv$ is the concatenation. $\mu$ is associative, you can check easily.
We actually have further a map $\eta: k\to A$ given by $a\mapsto a\emptyset $ where $\emptyset$ is the empty word in $X^*$. Now try to show that $\mu(\eta(a), V)=aV=\mu(V, \mu(a))$ where $V\in A$ is any vector and $a\in k$. This second map turns $A$ into a unital algebra. Hence we have a associative unital algebra...
For further reference Fundamentals of Hopf Algebras by Robert G. Underwood is a very good place to start learning about algebras, coalgebras, bialgebras, Hopf algebras, etc. You're question is kind of the beginning of it all. That book is introductory, very readable and simple.