The definition I'm using for a CSA over a field $k$ is the following:
A CSA over $k$ is a finite-dimensional associative $k$-algebra which is simple and has center precisely $k$.
My question concerns whether implicitly, every CSA is unital. If $A$ is a unital $k$-algebra, then there is a natural way to embed $k$ into the center of $A$ via $k \mapsto k \cdot 1_A$, but can it be done without a unit element?
Yes, central simple algebras are unital, since as you observe there is no obvious way to embed $k$ into $A$ when $A$ is merely a non-unital $k$-algebra.
Indeed, there might not even be any such embedding at all. Consider the smallest non-unital subring of $\mathbb{R}$ containing all rational multiples of $\pi$ (say), viewed as a $\mathbb{Q}$-algebra in the obvious way. This has no nonzero idempotent elements, so there is no embedding of $\mathbb{Q}$ into it.