If I understand correctly, for test functions $f$ we define
$$ \langle\delta, f\rangle = f(0)$$
and convolution is defined as
$$ \langle g * T, f\rangle = \langle T, g^- * f\rangle,$$
where $f$ and $g$ are test functions.
From this it follows that
$$ \langle g * \delta, f\rangle = \langle\delta, g^- * f\rangle = (g^- * f)(0) = \int g^-(-y) f(y) dy = \int g(y) f(y) dy = \langle g, f\rangle$$
so that $g * \delta = g$.
I don't see how any of this would imply that $ \delta * g = g$ as well. In fact, I don't even see how this expression makes sense, since $\langle \delta * g, f\rangle = \langle g, \delta * f\rangle $. Since $\delta$ is only defined as a distribution, I don't think $\delta * f$ has any meaning, let alone is a test function. Can someone clear this up for me?
To your question: Your definition of convolution is a map
$$ *:(\mbox{test functions})\times (\mbox{distributions})\to \mbox{distributions}. $$
So you're right, "$\delta * g$" does not make sense, as $\delta$ is not a test function.