Let us start by assuming that $\Bbbk$ is a field and $A$ is a $\Bbbk$-algebra. If $P$ is an $A$-bimodule (equivalently, a left $A^e$-module, where $A^e = A \otimes_\Bbbk A^{op}$) then being projective as an $A$-bimodule for $P$ implies being projective as a right/left $A$-module. This may be seen as following from the fact that, as $\Bbbk$-vector spaces, $$\left(\bigoplus_{s\in S}A^{op}\right)^{\dim_\Bbbk(A)} \cong A \otimes_\Bbbk \left(\bigoplus_{s\in S}A^{op}\right) \cong \bigoplus_{s \in S} A^e \cong \left(\bigoplus_{s\in S}A\right) \otimes_\Bbbk A^{op} \cong \left(\bigoplus_{s\in S}A\right)^{\dim_\Bbbk(A)}$$ and by suitably considering $A$-actions with respect to which the isomorphisms are linear (for instance, looking at the two right-most isomorphisms as left $A$-linear maps allows us to conclude that $P$ projective as $A$-bimodule implies $P$ projective as left $A$-module). The converse is clearly false: $A$ itself is always free as left/right $A$-module, but it is not projective as an $A$-bimodule in general.
Now, if $k$ is a commutative ring and $A$ is a $k$-algebra, can we still prove that being projective as $A$-bimodule for a certain $P$ implies being projective as left/right $A$-module for $P$?
Clearly, the foregoing argument does not work in this case and I am struggling looking for either an alternative proof or a counterexample.
$P$ is a direct summand of $A^e$, that is, of an $A^e$-module of the form $\bigoplus_{i\in I}A^e$.
As a left $A$-module, this thing is $\bigoplus_{i\in I} A\otimes_k A^{op}$.
In particular, the result holds for all $P$ if and only if it holds for $A\otimes_k A^{op}$.
In particular, this obviously holds whenever $k$ is a field (which is what you said), because any $k$-module is projective, so $A^{op}$ is projective as a $k$-module, so that $A\otimes_k A^{op}$ is projective as an $A$-module.
More generally, whenever $A$ is projective as a $k$-module, $A\otimes_k A^{op}$ is projective as an $A$-module.
But this doesn't hold in general. For an explicit example, take $k=\mathbb Z, A=\mathbb Z[X]/(2X)$.
Then, as an $A$-module, $A\otimes_k A^{op}$ is nothing but $A\oplus \bigoplus_\mathbb N \mathbb Z/2 [X]$ which is not projective, because $\mathbb Z/2[X]$ isn't (and the latter isn't because if you tensor it over $\mathbb Z[X]/(2X)$ with $\mathbb Z$, you get $\mathbb Z/2$ which isn't projective over $\mathbb Z$)