In Hatcher's book algebraic topology page 496-497:
the Steenrod algebra $\mathcal{A}_2$ is the algebra over $\mathbb{Z}_2$ formed by polynomials of non-commuting variables $Sq^1,Sq^2,\cdots$ modulo Adem relations.
the Steenrod algebra $\mathcal{A}_p$ is the algebra over $\mathbb{Z}_p$ formed by polynomials of non-commuting variables $\beta,P^1,P^2,\cdots$ modulo Adem relations.
What means polynomials of non-commuting variables $Sq^1,Sq^2,\cdots$? polynomials of non-commuting variables $\beta,P^1,P^2,\cdots$? Is it the same with free tensor algebra or different?
In Hatcher's book algebraic topology page 499:
Can we write $$\mathcal{A}_2=\text{Free Tensor Algebra}_{\mathbb{Z}_2}[Sq^I\mid I\text{\ admissible\ }]$$
$$\mathcal{A}_p=\text{Free Tensor Algebra}_{\mathbb{Z}_p}[P^I\mid I\text{\ admissible\ }]$$ or we cannot?
Let $X$ be a space. $H^*(X;\mathbb{Z}_p)$ ($H^*(X;\mathbb{Z}_2)$) is only a module over $\mathcal{A}_p$ ($\mathcal{A}_2$), or $H^*(X;\mathbb{Z}_p)$ ($H^*(X;\mathbb{Z}_2)$) is an algebra over $\mathcal{A}_p$ ($\mathcal{A}_2$)? Is there some product?
$\DeclareMathOperator\Sq{Sq}$Yes, that's what it means here. The algebra $\mathcal{A}_2$ is the quotient of the free tensor algebra generated by $\Sq^1, \Sq^2\dots$ quotiented by the ideal generated by the Adem relations. Similarly for $\mathcal{A}_p$, $p>2$.
However be careful with the admissible thing. What Hatcher shows is that every element of the algebra can be written as a sum of admissible monomials. In other words he shows that this set of admissible monomials generate the algebra as a vector space. However there are multiplicative relations between these monomials, because eg. $\Sq^1$ and $\Sq^2$ are admissible monomials, and their product $\Sq^2 \Sq^1$ is still admissible. So $\mathcal{A}_2$ is not a free algebra on the admissible monomials, because we have the relation $\Sq^2 \Sq^1 = \Sq^I$ (where $I = (2,1)$). The point is that they form an additive basis (ie. as a vector space), not a "multiplicative basis".
Cohomology is not (that I know of) an algebra over $\mathcal{A}_p$, though, at least for the cup product. We have the Cartan formula: $$\Sq^n(x \cup y) = \sum_{i+j=n} \Sq^i x \cup \Sq^j y.$$ Whereas we would have wanted $\Sq^n (x \cup y) = (\Sq^n x) \cup y = x \cup (\Sq^n y)$ in an algebra over $\mathcal{A}_2$. The same problem arises with $P^n$.