In Cox's Ideals, Varieties, and Algorithms, he defined a $k$-algebra is a ring which contains the field $k$ as a subring. Also: A $k$-algebra is finitely generated if it contains finitely many elements such that every element can be expressed as a polynomial (with coefficients in $k$) in these finitely many elements.
Question: According to this definition, $k[x_1, x_2, ..., x_n]$ is a finitely generated $k$-algebra but it is not a finitely generated $k$-module. Am I right? I have this question because being a $k$-algebra is also a $k$-module.
Yes, you're correct.
The polynomials of the form $x_1^{a_1}\cdots x_n^{a_n}$ form a $k$-basis for $k[x_1,\dots,x_n]$, and there are infinitely many of them.