What makes induction (over natural numbers) a valid proof technique? Is $$ \dfrac{ P(0) \quad \forall i \in \mathbb{N}. P(i) \Rightarrow P(i+1) }{ \forall n \in \mathbb{N}. P(n)} $$ just taken for granted as a proof rule, or can it be derived from more foundational axioms?
Similarly, can the principle of induction over well-founded sets be derived from something more foundational, or does it have to be assumed to be a valid inference rule?
On
In Peano arithmetic, induction is an axiom schema: your proof technique is simply $a, (a \implies b) \vdash b$, where the hypothesis of your proof technique is $a$, and $(a \implies b)$ is an instance of the induction schema.
On
One point of view is that "the natural numbers satisfy induction" is part of what we mean when we're talking about the natural numbers; that is, part of the definition of "the natural numbers" should be "those things that satisfy induction." This is just a slightly more sophisticated version of "the natural numbers are what you get when you start with $1$, then add $1$ to it, then add $1$ to it, then..."
If someone tells you that badgers are white, 200 feet tall, and glow in the dark, they're not even wrong about badgers: whatever they mean by badger, it isn't what you mean by badger. Similarly, if someone tells you that the natural numbers don't satisfy induction and also include $-3$ and $\frac{5}{7}$, then they're not even wrong about the natural numbers: whatever they mean by natural number, it isn't what you mean by natural number.
People think of axioms as laws you have to follow or true things you have to assume and I think neither of these perspectives is correct. It's more accurate to think of axioms as a way to agree that we're talking about the same thing.
On
I'll try to give a "philosophical" answer:
If you take the so-called "Platonic" viewpoint which many mathematicians seem to share more or less, then mathematics is something which exists in some realm outside of us and can be "examined" by our minds. From this point of view the natural numbers are a reality that everybody experiences in the same way and induction is something that's "obviously true" about them. (Or, in other words, you have to "believe" it.)
If, on the other hand, you take a more formalist point of view and adhere to Bertrand Russell's claim that mathematics is essentially a collection of statements of the form $A \Rightarrow B$, then before you start investigating a subject you try to find a couple of axioms that describe your subject as concisely as possible. If you do it this way, then the principle of induction (or something which is equivalent to it) is essentially always an axiom, i.e. something that can't be proved.
One set of axioms that can in principle be used as a basis for almost all of mathematics (and is what you normally use without thinking about it unless you are either an ultrafinitist or working on foundations) is ZFC which includes an "axiom of infinity" from which the existence (!) of the set of natural numbers and then the principle of induction can be derived.
In some areas of mathematical logic you're specifically interested in "weaker" axiom systems to see what you can (and what you can't) prove with them alone. This is were systems like the Dedekind-Peano axioms (also of great historical interest) come into play which also include an axiom (or sometimes a whole schema of axioms) for induction.
Or, to put it more strikingly: The principle of induction says something about infinitely many objects which means that you can neither prove nor "check" it with finite means (and that's all we mere mortals can do). The only way to "prove" it is to fall back to some other proposition about infinitely many objects which again can't be proved.
There is a property of the natural numbers called "well-ordering". It says that any non-empty subset of $\mathbb{N}$ has a smallest element. We can use this to prove induction.
Let $S = \{ n \in \mathbb{N} \mid \lnot P(n) \}$. Assume it is non-empty. Then there is a least element; call it $k$. Since $P(1)$ is true, we know that $k \ne 1$. Therefore*, $k > 1$, so $k - 1 \in \mathbb{N}$. But since $k - 1$ is less than $k$, which was the minimum element of $S$, we know $k - 1 \notin S$. Therefore, $P(k - 1)$ is true. But our other hypothesis tells us that $P(k - 1 + 1) = P(k)$ is also true. This is a contradiction, so our assumption that $S$ was non-empty must be false. This means that $P(n)$ is true for all $n \in \mathbb{N}$.
The proof for transfinite induction on a well-ordered set is nearly identical. The only difference is that once you have $k$, you state all elements less than $k$ satisfy $P$, not just $k - 1$. But that is exactly your inductive hypothesis, so you get the desired contradiction.
*We haven't shown that all natural numbers are $\ge 1$, but that can also be shown with well-ordering. Briefly: Take the set of all $n$ such that ($0 < n < 1$). Assume it's non-empty. We take the smallest element, $k$, and square it to make a smaller one. Since $0 < k^2 < 1$, this contradicts the minimality of $k$. Therefore, that set is empty, and there are no natural numbers less than $1$.