The definition of a linearly independent set I have been given is: A set S is linearly independent if every finite subset of S is linearly independent.
Do there exist any sets S such that all finite subsets are linearly independent, but contain an infinite subset that is linearly dependent? If not, why include the 'finite' in the definition?
First, there is the usual definition of linear independence of a finite set of vectors: namely, $\{v_1,\dots, v_k\}$ is linearly independent if $\lambda_1v_1+\dots +\lambda_kv_k=0$ implies all $\lambda_i=0$.
Then, one can extend it for infinite sets, say, by the given definition.
Note that vector addition, hence also linear combination, is defined only for finitely many vectors, so if a vector $u$ is linearly dependent on a set $S$, then it means it is a linear combination of finitely many of them.