So I have the two following definitions:
An $R$-algebra $S$ is said to be finite over $R$ if it is finitely generated as an $R$-Module.
An $R$-algebra $S$ is said to be finitely generated if $$S \cong R[x_{1},x_{2}, \dots x_{n}]/I$$ for some $n$ and some ideal $I$.
Do these two definitions mean the same thing?
Not at all: $R[x]$ is generated by $x$ as an $R$-algebra, but it's a free $R$-module with an infinite basis.
Actually a finitely generated $R$-algebra is finite if and only if its generators (as an algebra) are integral over $R$.