Wikipedia says that in certain situations, $Ext^\ast_A(R,R)$ becomes a ring, such as when $A$ is an augmented $R$-algebra, but the outline is too sketchy for me to understand.
I can't find this in Weibel. Isn't $Ext^\ast_A(M,M)$ a ring (even an $\mathbb{N}$-graded algebra over center $Z(A)$) for any algebra $A$, via the splicing (concatenation) of extensions (exact sequences) $$0\to M\to E_k\to\ldots\to E_1\to M\to0\text{ and }0\to M\to E'_l\to\ldots\to E'_1\to M\to0,$$ $$\text{producing }0\to M\to E_k\to\ldots\to E_1\to E'_l\to\ldots\to E'_1\to M\to0\in Ext^{k+l}_R(M,M)?$$
A more detailed sketch or a reference would be welcome. How can this multiplication be defined when we interpret $Ext^n_A(M,M)$ as $H_n(Hom_A(P_\ast,M))$ for some projective resolution $P_\ast\to M$?
If I understand correctly, this makes the group and algebra and Lie algebra cohomology $\bigoplus_{n\geq0}Ext^n_{R[G]}(R,R)$ and $\bigoplus_{n\geq0}Ext^n_{A\otimes A^{op}}(R,R)$ and $\bigoplus_{n\geq0}Ext^n_{U(\mathfrak{g})}(R,R)$ into rings, correct?
Can some similar construction be made for $Tor$?
You should think of $\text{Ext}^{\bullet}_A(R, R)$ as a shadow of a more fundamental operation, namely taking derived endomorphisms $\text{RHom}_A(R, R)$ of $R$, regarded as an object in the derived category of $A$-modules. Here the ring structure is obvious; the multiplication comes from composition. More generally, there is a composition map $\text{RHom}(A, B) \times \text{RHom}(B, C) \to \text{RHom}(A, C)$, which you can write down by replacing $A, B, C$ with projective resolutions and then composing chain maps.
The situation for $\text{Tor}$ is different; you should also think of $\text{Tor}$ as a shadow of a more fundamental operation, namely taking derived tensor products $M \otimes_A^L N$, but even in the underived setting the tensor product of two modules has no reason to acquire a ring structure. One should only expect a ring structure if $M$ and $N$ are themselves already $A$-algebras, which happens e.g. if $A$ has an augmentation $A \to R$ and $M = N = R$.