Let $A$ be a $k-$algebra with $k$ field. Suppose $A$ is finite dimensional over $k$.
$\textbf{Q:}$ Is it possible to find a basis $B$ of $A$ over $k$ s.t. the basis set is closed under multiplication in general? It seems this is true in finite quiver path algebra quotient by admissible ideals or finite group algebra over field $k$. If not, what is the counter example?
No, the existence of such a basis is an extremely strong condition. For instance, notice that for any $x\in B$, the powers of $x$ must be eventually periodic, since they are all elements of the finite set $B$. In particular, if $A$ is a field extension of $k$, this implies every element of $B$ is a root of unity, and so $A$ must be spanned by the roots of unity it contains. So for instance, if $k=\mathbb{Q}$ and $A=\mathbb{Q}(\sqrt{2})$, this is not possible since the only roots of unity in $A$ are $\pm 1$.