According to Dummit and Foote's definition, an $R$-algebra ($R$ is a commutative ring with identity) is a ring A with identity together with a ring homomorphism $f: R \rightarrow A$ mapping $I_R$ to $I_A$ such that the subring $f(R)$ of $A$ is contained in the center of A.
Why does it necessarily follow that any ring with identity is a $\mathbb{Z}$-algebra? What would be the homomorphism?
Hint: where do you send $2 = 1 + 1$?