Many arguments about the foundations or philosophy of mathematics centre on the question of whether or not there exist objects or entities (such as certain sets) which are not "finite".
(For instance, Doron Zeilberger, although he is fond of April Fool jokes, does not seem to be joking here, or here.)
For such arguments to take place at all, the participants must share some understanding of what the word "finite" means.
I have never taken part in any such argument, but if I were to have to do so, I would nervously have to admit that I do not know what this shared understanding is, and I would ask if the others would mind spelling out (if only for my benefit) what it is that they think they are arguing about.
Just to anticipate two possible lines of ensuing discussion (assuming that I wasn't just greeted by stunned or embarrassed silence):
If (assuming for the moment that there was no dispute as to what a "set" is) someone were to say that a set is finite if and only if it can be put into one-to-one correspondence with a set of the form $\{1, 2, \ldots, n\}$ for some natural number $n$, this would obviously be open to the objection that it takes for granted a common understanding of the existence of a unique set of natural numbers, known to all the participants in the discussion. It might perhaps also be objected to because it apparently implies performing a possibly non-terminating search for such a number $n.$ But even if that is not a problem, it surely cannot be the case that a person cannot even be said to know what the word "finite" means unless they already accept the existence of the (uniquely defined) infinite set of all natural numbers.
(Zeilberger, for one, would presumably be inclined to make some objection(s) along these lines.)
If, on the other hand, someone were to put forward Dedekind's definition that a set is finite if and only if it cannot be put into one-to-one correspondence with a proper subset of itself, this could be objected to because it seems to imply that you can only directly show a set to be finite by proving something about the collection of all mappings of that set into itself, whereas this can hardly be what anybody has in mind when they insist that all mathematical objects must be finite, therefore it cannot be the agreed-upon common understanding of what the word "finite" means.
(Tarski's definition that a set is finite if and only if every non-empty collection of its subsets contains a maximal element is open to a similar objection. So too is Staekel's definition that a finite set is one that can be doubly well-ordered.)
So, some other definition of "finite" would have to be agreed upon; but what might it be?
Of course, the participants, whatever their other disagreements, might silently agree upon silence as the only appropriate response to such a ridiculously naive and ignorant question; but what if it had to be discussed, say for the benefit of a child, whose naivety and ignorance could be excused?
Your question strikes me as odd, because it asks "What do people mean by finite?" while simultaneously discussing two perfectly good definitions of the term.
A set $S$ is finite if there exists a bijection from $S$ to a section of the natural numbers. Equivalently, $S$ is finite if every injection from $S$ to itself is a bijection. These definitions are logically equivalent, so it makes no difference which one you use as a definition and which one you prove as a theorem.
You raise the objection that the natural numbers definition requires a common understanding of what the natural numbers are. That's a reasonable point, which is why mathematicians typically either define the natural numbers axiomatically or construct them as part of a larger axiomatic system. Historically, the natural numbers were defined axiomatically using the Peano axioms, but in the modern foundations of mathematics they can be constructed explicitly using the ZFC axioms for set theory. What this means is that the only common notions that are required to discuss mathematics precisely are the rules for symbolic logic.
Now, one can raise the objection that the axioms of ZFC may be inconsistent, in which case our entire discussion of mathematics is, from a formal point of view, entirely fruitless. Presumably this is what radical finitists like Zeilberger believe. Though we have no way to prove that the ZFC axioms are consistent, they have been working well so far, so the onus is on the finitists to demonstrate an inconsistency.