In studying GIT I encounter the same problem at multiple occasions of confusing group schemes for abstract groups . There is a natural example:
Take an affine scheme $X=Spec(A)$ and an (affine) group scheme $G=Spec(B)$ over some field $k$ (not necessarily algebraically closed) acting on $X$. That is a k-morphism $\alpha:G\times X\to X$ or equivalently a morphism
$$ \alpha^*: A\to A\otimes B $$ such that the action (resp. coaction) conditions are fulfilled.
With this set up I often see the notation $Spec(A^G)$ (not only for finite groups!) and I really don't see what this means.
My interpretation: In these lecture notes the following is indicated (Remark 3.6.):
For $g\in G$ and $f\in A$, we have $\alpha^*(f)=\Sigma f_i\otimes h_i$. Then $g$ acts as follows
$$ f\mapsto \Sigma h_i(g)f_i .$$
But this expression only makes sense to me if we speak about $G(k)$ and not $G$. Thus one should rather write $Spec(A^{G(k)})$ instead of $Spec(A^G)$.
Am I missing something here or is the above approach correct and this is simply an abuse of notation?
You have the notion of a group scheme $G$ acting on some scheme $X$. But you also have the notion of any abstract group $G$ acting on a scheme $X$, that is, for every $g \in G$ there are morphisms $f_g: X \to X$ satisfying the axioms of a group action. This action is the same as the action of the constant group scheme $G$ on $X$.
Now let $G$ be any finite group acting on an affine scheme $Spec(A)$ and $A^{G} \subset A$ denote the subring of $G$–invariant elements. Then the canonical morphisms $(p,p^{\sharp}): Spec(A) \to Spec(A^G)$ induce
I hope it's now clear where the notation comes from.