Addition and multiplication may be defined in two ways, one specific and one general:
Addition
- specific: addition is repeated incrementation.
This is specific and sub-optimal as while $2 + 4$ is defined, $2 + 1.3$ is not, as you cannot repeat an action $1.3$ times.
- general: addition is shifting the number by the number.
For example $2 + 1.3$: Keeping the position of the number line fixed, I move the number $2$ $1.3$ units in the positive direction (to the right). This makes physical sense and you can actually build the number spaces (lines for simplicity) and move them around.
For complex numbers, the imaginary component is just moving up and down instead of right and left for the real component.
Multiplication
- specific: multiplication is repeated addition
Same as above, how do I multiply by a decimal number?
- general: multiplication ($a * b$) is stretching the segment that goes from $0$ to $a$ until the segment from $0$ to $1$ becomes as long as $b$
Again, this works for decimals and makes physical sense as you can imagine taking a rubber segment and pulling the extremes apart to stretch it and compare it to the original number line.
Multiplying by an imaginary is rotating instead, that you can also easily imagine doing (or actually do) to a plane.
Exponentiation
- specific: exponentiation is repeated multiplication
Same as above, how do I exponentiate by a decimal number?
general: ???
- What is a physical representation of exponentiation that gives a clear yet rigorous and general description for it?
- Could you also talk about exponentiation by complex numbers?
- In other words, if I gave you a rubber complex plane (or more simply, real number line) and asked you to raise a number to the power of another number, what would you do?
- How would you deform it to represent the operation in an intuitive way?
There are actually two answers to your question, because exponentiation is the first hyperoperation that is not commutative. Addition and multiplication are commutative, so they can only be curried in one way. To explain what I mean better, look at multiplication. We have $f(x, y) = x \times y$. From this, we can define "multiplication by $x$" as $m(x) = \lambda y \rightarrow f(x, y)$ i.e. it is the function that takes a $y$ and maps it to $x \times y$. But we could also have chosen $m(x) = \lambda y \rightarrow f(y, x)$. However, since multiplication is commutative, it doesn't matter which we choose. It's the same for addition.
OK, so what about exponentiation? Here, there is a big difference: exponentiation is not commutative. This may be obvious, but I'll give an example anyway: $3^2 = 9 \neq 8 = 2^3$. Right, so we've established that commutativity is lost. So how does this change things? Well, it means that you now have to consider two functions. Setting a constant $a$, we can either look at the function $p(x) = x^a$ or $g(x) = a^x$. Generally speaking, the latter is called an exponential and the former a polynomial, but both use exponentiation. The difference is that in a polynomial, the base varies and the exponent remains fixed, whereas in the latter it is the base that remains fixed while the exponent varies. So to answer your question, we need to consider these two completely different functions. To keep things simple, I'll just focus on the real line.
Polynomial function: Consider the function $p(x) = x^a$. What does it do to the real line? We'll focus just on the positive real line: the negative real line is very interesting in its own right, but we end up with complex values for non-integer $a$, so to keep things simple I'll omit discussion of this for now. Right, so for this function to be interesting we have $a \neq 0$, otherwise we just have $p(x) = x^0 = 1$; in other words, all points get squashed onto the point $1$. If $a$ is negative, then we must consider first what the function $\lambda x \rightarrow \frac{1}{x}$ does, so we'll leave this too for now. So focus on a positive $a$. What does it do? There are three cases to consider:
Exponential function: We now explore $g(x) = a^x$. Again, I'll ignore negative $a$ in order to avoid stepping into the domain of complex numbers. I'll also avoid $0^x$ because this is just $0$ everywhere except at $0$, where it's $1$, so it's not really a smooth transformation. That leaves us with a positive value for $a$. Furthermore, we'll just consider $a>1$, because $a=1$ is the boring constant function, and $0 < a < 1$ gives the mirror image of some function $a > 1$, so we just study the latter and note the symmetry. So what does $a^x$ look like over the real number line, for some $a > 1$?
For $x < 0$, we are looking at the entire negative real axis. This entire axis gets squished onto the open interval $(0, 1)$. No value gets mapped to $0$. We can imagine this as the exponential function being an incredibly strong vacuum cleaner capable of sucking up the entire infinitely long negative real axis into the open unit interval.
$a^0 = 1$, i.e. the exponential function always maps the value $0$ onto $1$. This is interesting, because it is independent of the value $a$. It is as if the mapping from $0$ to $1$ is more concrete than the rest, which vary with $a$.
For $x > 0$ the exponential function stretches the real number line in a non-uniform way. There is a subtle but important difference to multiplication. With multiplication, you stretch all points uniformly i.e. you make all distances scale up by the amount $a$. With exponentiation, you stretch more and more the further to the right you go. That is, points get stretched further and further apart the further out to the right you go. Slightly more precisely, imagine multiplying $x + d$ by $a$. You first stretch $x$ out by $a$, then relative to that point, you translate out (add) by $d$, and then stretch it by $a$. Note that the stretching of $d$ by $a$ is independent of $x$ i.e. you only stretch the $d$, not the rest. This is simply the distributivity law for addition and multiplication. Exponentiating $a$ to the $x + d$ is different. You first exponentiate $x^a$, then you stretch this by $x^d$. In other words, the amount by which you stretch in the exponential function, is relative to how much you've already stretched, whereas with multiplication, the stretching is constant.
There's a lot more I could have covered in this answer, particularly extending this reasoning to the complex domain and giving more precise mathematical definitions. However I chose instead to focus in on some special cases in detail, and visualising where things go, rather than trying to cover everything. In light of this there is plenty of scope for others to answer this question and cover these additional concepts. I see Ante Paladin has already made some progress towards this end.