Wikipedia says that a graded $A$-algebra is just a graded $A$-module that is also a graded ring.
Question: when one says then "finitely generated graded $A$-algebra", does one mean that every element $s$ can be written as $s=\sum^N_{i=0} a_i g_i$, where $\{g_i\}_{i=1}^N$ is a generating set, or that it can be written as $s=\sum^N_{i=0} a_i \prod_{j\in I} g_j$?
Ravi Vakil proposes in an exercise:
4.5.D. (a) Show that a graded ring $S_\bullet$ over $A$ is a finitely generated graded ring (over $A$) iff $S_\bullet$ is a finitely generated graded $A$-algebra, i.e., generated over $A=S_0$ by a finite number of homogeneous elements of positive degree.
What I don't understand is how putting the word "algebra" instead of "ring" makes the difference that the former is supposed to mean generated over homogeneous elements and the latter by "any elements". Doesn't, in general, an $A$-algebra simply mean a ring over $A$, i.e. a ring into which $A$ maps (possibly not injectively), so that an $A$-action is defined?
Serge Lang in his book "Algebra", defines an $A$-algebra to be any ring $B$ together with a ring homomorphism $f:A\rightarrow B$. This corroborates your interpretation.
A finitely-generated $A$-algebra is practically understood as a quotient ring of a polynomial ring in finitely many indeterminates. In other words it has the form $S=B[X_1,\dots,X_n]/I$, where $B$ is a ring, $I$ is an ideal of $B[X_1,\dots,X_n]$ and there exists a ring homomorphism $A \rightarrow B/(I \cap B)$. So far, this is independent of the grading.
Now an $A$-algebra $S$ with a $\mathbb{Z}_+$-grading (for simplicity), is a ring $S$ together with a ring homomorphism $f:A \rightarrow S$, such that $S$ can be written $S=\oplus_{i \ge 0} S_i$, and the multiplication operation of $S$, $S \times S \rightarrow S$, satisfies $S_i \times S_j \rightarrow S_{i+j}$.
I am not expert enough, but it seems to me that the phrases "graded ring $S$ over $A$" and "graded $A$-algebra $S$" indicate the same thing. This is true if we interpret the phrase "$S$ is a ring over $A$" to mean simply that we have a ring homomorphism $A \rightarrow S$, but then using Lang's definition $S$ is an $A$-algebra.
Finally, i believe that the point of the exercise you quote is to show that for a finitely-generated graded $A$-algebra, we can take the generators to be homogeneous elements.