Some Motivation:
I really have no understanding what happens when we tensor two $k$-algebras. Tensoring by $\otimes_{\mathbb Z} \mathbb Q$ for an abelian group literally "takes away torsion". How should one interpret, for example $\otimes_{\Bbb Q} \Bbb C$ for $\Bbb Q$-algebras?
I am not interested in general base change results that preserve finiteness of algebras etc. which is not particular to the nature of the field $k$ involved. I want a few concrete examples to keep in mind. So I am mainly interested when $k=\Bbb Q, \Bbb C$.
So I am wondering about the property of being integral. Let $A$, $B$ be integral $k$ algebra.
What are examples/non examples when $A \otimes_k B$ is integral, when $k= \Bbb Q, \Bbb C$.
It would help if one provides the intuition and explanation in approaching this problem.
I rephrase your question slightly because I think this is more what you are after:
The point is: when viewed as a $\mathbb{Q}$-algebra the algebra $A \otimes \mathbb{C}$ is an infinite-dimensional moloch that bears almost no discernible relationship to $A$, while viewed as a $\mathbb{C}$-algebra the algebra $A \otimes_\mathbb{Q} \mathbb{C}$, which in that context is often written $A_\mathbb{C}$ looks very much like $A$. It has the same dimension, we can even take it to have the same basis (as a vectorspace) and describe its multiplication in terms of a table that gives the products of these basis elements - thus giving the appearance that the algebra $A$ can be described independent of the base field and $A$ and $A_\mathbb{C}$ are essentially the same thing.
Of course this appearance is misleading: for instance we will often have that $A \otimes_{\mathbb{Q}} \mathbb{C}$ is non-integral while $A$ is integral. This is, I believe, what your question is about.
Here is an answer:
It follows that for finite dimensional $A$ algebras over any field $k \subset \mathbb{C}$ we have that integrality is destroyed when moving to $A \otimes \mathbb{C}$, unless $A$ was one-dimensional (i.e. equal to $k$) to begin with.
I write two concrete examples.
1) $k = \mathbb{Q}$, $A = \mathbb{Q}(\sqrt{2})$.
Clearly $A$ is a division algebra. To see that $A_\mathbb{C}$ is not we need to distinguish between the element $\sqrt{2} \in A$, which is a basis vector of $A$ and $A_\mathbb{C}$ and the element $\sqrt{2}$ in $\mathbb{C}$, which is a scalar from the perspective of $A_\mathbb{C}$. To be completely on the safe side we write $\mathbb{v}$ for the vector $\sqrt{2} \in A$ and $\lambda$ for the scalar $\sqrt{2} \in \mathbb{C}$.
Now compute $(\mathbf{v} \otimes 1 - 1 \otimes \lambda)(\mathbf{v} \otimes 1 + 1 \otimes \lambda)$. By strange product we have that this equals $(\mathbf{v} \otimes 1)^2 - (1 \otimes \lambda)^2 = (\mathbb{v}^2 \otimes 1^2) - (1^2 \otimes \lambda^2) = (2 \otimes 1) - (1 \otimes 2) = 2(1 \otimes 1) - 2(1 \otimes 1) = 0$. For the second-to-last equality I switched back to the definition of $A \otimes_{\mathbb{Q}} \mathbb{C}$ as a $\mathbb{Q}$-algebra rather than $\mathbb{C}$-algebra so that we can shamelessly pull out scalars from $\mathbb{Q}$.
2) $k = \mathbb{R}$, $A = \mathbb{H}$ the quaternions. This is, famously, a division algebra. I include this example because it is non-commutative. We could also use a quaternion division algebra over $\mathbb{Q}$ (in fact there are infinitely many of them) but I go with $k = \mathbb{R}$ because this quaternion algebra is better known and even has its own name. Now $\mathbb{H}_\mathbb{C} = \mathbb{H} \otimes_\mathbb{R} \mathbb{C}$ when considered as a $\mathbb{C}$ algebra is isomorphic to $Mat(2, \mathbb{C})$, the algebra of $2 \times 2$-matrices! You need to think a bit about why this is true, but much easier is to check that, again integrality is lost.
Something else is lost too: let $B = Mat(2, \mathbb{R})$. Then $A \neq B$ but $A_\mathbb{C} = B_\mathbb{C}$!
Here is a different answer:
Example: $A = k[X]$, the algebra of polynomials. Then $A_\mathbb{C} \cong \mathbb{C}[X]$.