My question actually narrows down to the meaning of mathematical principle. While I'm looking for some principles, they usually have their proofs, so I thought "principle" has the same meaning as "Theorem", but most explanations just state that principles are just intuitively obvious rules. Moreover, I've found a principle given without proof, namely the "Comprehension principle" by Frege, but it has been identified as a paradox by Russell. So, or but, it still remains leery to me.
Does every mathematical principle have a proof?
Proofs begin with axioms. So what are the axioms we are using?
Given certain axioms, sometimes we can prove certain principles. Other times, we assume these principles as axioms, so their proofs become trivial.
Let me give a better example. The principle of mathematical induction is a theorem of the axioms of $\sf ZFC$. In fact, set theory proves much stronger principles of transfinite induction, and well-founded induction. And the mathematical induction we apply to the natural numbers are an easy corollary of either one.
On the other hand, if we want our settings not to be the axioms of set theory, but just the axioms of arithmetic, then it is common to assume the principle of mathematical induction as an axiom (either a first-order schema, or a second-order axiom).
Of course, we can devise other axioms which are not induction, that can be used to prove the principle of induction. This is also the case with set theory. We can replace some of the axioms by what is essentially a formulation of transfinite induction, and then we can show that the axioms we removed are still provable.
And sometimes we work in a system where mathematical induction is just not provable, or maybe even plain false.
To sum up, principles are like axioms. These are names for statements that we view, philosophically, as somewhat reasonable behavior of certain mathematical objects.
We can prove some mathematical principles from certain axioms, or we can assume them as axioms altogether. And we can even devise principles which are incompatible one with another. Because mathematics does not have some absolute context, but rather a lot of different contexts where the basic, foundational, axioms can be different.
And of course, if you assume principles as axioms, then they have a trivial proof.