There is a theorem (VIII.2.3) in Mac Lane's Homology that reads:
Let $k$ be a commutative ring. Let $R,S$ be $k$-algebras, and let $U$ be a $k$-differential graded algebra. Suppose there is a morphism $U\to R$ of differential graded algebras ($R$ concentrated in degree zero with trivial differential) such that $U$, regarded as a complex, is a projective resolution of the $k$-module $R$. Then the canonical isomorphism of $k$-graded modules $Tor_*^k(R,S)\cong H_*(U\otimes_k S)$ is an isomorphism of graded $k$-algebras (where $S$ is considered with trivial differential, and thus $U\otimes S$ is a $k$-dga so its homology is a graded $k$-algebra).
In the next section he goes on to treat products in Ext but I'm surprised by the lack of an analogous theorem. Sure, when we treat products in Ext we need more hypotheses in order to have internal products that beget an algebra, but sometimes we have them. I wonder if the following is true:
Let $k$ be a commutative ring, $R$ be an augmented $k$-algebra. We turn $k$ into an $R$-module by pullback of the augmentation $R\to k$. In this setting, $Ext^*_R(k,k)$ is a graded $k$-algebra.
Let $U$ be an $R$-dga and suppose there is a morphism $U\to k$ of dgas, where $k$ has trivial differential, such that $U$ is a projective resolution of the $R$-module $k$. Then the canonical isomorphism $Ext^*_R(k,k)\cong H_*(Hom_R(U,k))$ is an isomorphism of graded $k$-algebras.
Similarly, let $V$ be an $R$-dga and suppose there is a morphism $k\to V$ of dgas such that $V$ is an injective resolution of the $R$-module $k$. Then the canonical isomorphism $Ext^*_R(k,k)\cong H_*(Hom_R(k,V))$ is an isomorphism of graded $k$-algebras.
I don't really see why it wouldn't be, but I'm hoping for input for people more experienced with homological algebra.