We can construct a division algebra without multiplicative inverses?
In the article the octonions (http://math.ucr.edu/home/baez/octonions/node2.html), this is possible by taking the quaternions and modifying the product slightly, setting $i^2 = -1 + \epsilon j$ for some small nonzero real number $\epsilon$ while leaving the rest of the multiplication table unchanged.
No, that is even implicitly required in the definition you're citing:
Earlier:
If $a$ is nonzero, then left multiplication by $a$ is an injective endomorphism of a finite dimensional vector space, and hence it is also surjective. This means there exists a $b$ such that $ab=1$.
The whole point of a division algebra is that you can divide...