I am struggling to understand how to think about cohomology algebra of DGA.
In particular, let $H^*$ be cohomology algebra of a differential graded algebra $(A,d)$.
I know $(A,d)$ is the cochain complex: $$\dots\longrightarrow A^{k-1}\longrightarrow A^k\longrightarrow\dots$$ With $d:A^p\longrightarrow A^{p+1}$ for any $p$,
but then what is $H^*(A)$?
Secondly, how to think of the map $H^*\times H^*\longrightarrow H^*$ (here $\times$ is cross product)?
$H^n(A) = \ker (d^n)/\mathrm{im }(d^{n-1})$, as usual, and it can be arranged into a graded group $H^*(A) = \bigoplus_n H^n(A)$.
For multiplication, notice that being a differential graded algebra implies :
1) If $u,v$ are cocycles, then so is $uv$.
2) If $u,v$ are cocycles that differ by a coboundary, and $w$ is a cocycle, then $uw$ and $vw$ differ by a coboundary, and similarly for $wu$ and $wv$.
It follows that multiplication is well-defined on $H^*(A)$ by the formula $[u][v] = [uv]$: $uv$ is indeed a cocycle by 1), and by 2) changing the representatives for $[u], [v]$ doesn't change the class $[uv]$.
Another way to see that is to note the following : the formula relating the differential to the multiplication on $A$ essentially means that $m:A\otimes A\to A$ is a chain map, where $A\otimes A$ is the usual tensor product of cochain complexes, with the usual sign on the differential.
Therefore it induces a graded map $H^*(A\otimes A)\to H^*(A)$, and then you have a canonical map $H^*(A)\otimes H^*(A)\to H^*(A\otimes A)$: it sends $[u]\otimes [v]$ to $[u\otimes v]$ : you have to check that it is well-defined, but that's not very complicated either (it's the sale argument as above, which is why it may not be more efficient)