I've been thinking about this for quite a while but I cannot seem to find an example of
If $k$ is a commutative ring of finite global dimension and I'm looking for a strictly not-commutative $k$-algebra $A$, which is $k$-flat and there is an ideal $I$ in $A$ such that $A/I$ is a regular commutative ring which is $k$-flat also....
Edit: and I is generated by an element which is not a unit not a zero divisor and it commutes with all the elements of A...
(the flatness condition seems to really restrict my possibilities....)
Let k be a field, let A be the enveloping algebra of the three-dimensional Heisenberg Lie algebra g, and let I be the ideal of A generated by any nonzero element of the center of g.
This satisfies all your conditions.