Considering Peano axioms we'll define addition, multiplication and exponentiation operations.
We can then prove that addition and multiplication operations are commutative and associative. The proof for addition operation is pretty straightforward but for for multiplication it a bit more difficult. For example, to prove associativity, we have to prove distributivity first.
Moreover, for the definition itself it is not obvious that commutativity should take place, because the definition is not symmetric.
$a \cdot 0 := 0$
$a \cdot S(b) := a \cdot b + a$
Here $S(b)$ is the "next" function from Peano axioms
This definition is not symmetric with respect to $a$ and $b$, so commutivity is not obvious.
For exponentiation operation neither commutivity nor associativity take place. And I am very curious about the logical explanation why that happens
NOTE: The explanation I am looking for is somehow similar to a Abel–Ruffini theorem which explains WHY there is no an algebraic solution for a general polynomial equation
I'm not sure how to explain it using the Peano axioms other than that it's not, that this follows from the axioms and the definition and there's nothing more to it.
On the other hand, if you need some intuitions, you can find them by interpreting number $n$ as $\big|\{0,1,2,...\ldots,n-1\}\big|$ and observing that $a = |A|$ and $b = |B|$ implies $a \cdot b = |A \times B|$. No wonder multiplication is commutative, because the roles of sets $A$ and $B$ are symmetric. Similarly for $(a\cdot b)\cdot c = a \cdot (b \cdot c)$ which follows from $(A \times B) \times C \equiv A \times (B \times C)$ (I'm assuming you have the intuition why the last one is true).
Yet, for exponentation we have $a^b = |B \to A|$, i.e., the size of the set of functions from $B$ to $A$ (it is sometimes also written as $A^B$). However, note that function of the form $B \to A$ is much different than a function $A \to B$, for example if $B$ is empty and $A$ is not, then the latter cannot exist. The two operands on both sides of the arrow have different roles, so no wonder exponentiation is not commutative.
Also $(A \to B) \to C$ differs a lot from $A \to (B \to C)$. To see this, observe that the first one expects as input a rather huge amount of information compared to the second which is equivalent to $(A \times B) \to C$ (the formula $(a^b)^c = a^{b\cdot c}$ is not a coincidence). Thus, it's not surprising that the exponent is not associative.
I hope this helps $\ddot\smile$