So what I have to prove is that pushdown automaton with a limited stack size $k \in N > 0$ describes exactly the regular languages.
[Edit:]
I know that regular languages don't need any memory or stack, so the stack size of the PDA shouldn't matter. I just don't know how to go from here. Maybe one possible way would be showing it for stack size k = 1 and then prove by induction? Would I use a regular language as an example for that?
There are two things that you need to show here.
Every regular language is accepted by a pushdown automaton with stack size $k$. This is trivial, since, as you pointed out, DFAs can do this without a stack at all.
Every language accepted by a pushdown automataon $M$ with stack size $k$ is regular. This can be done by constructing a DFA $M'$ that accepts the same language as $M$. For this, consider the set of possible states that $M$ can be in (together with the possible states of the stack). Note that this is finite, and in the DFA, have one corresponding state for every pair of (state of $M$, state of stack). You should be able to figure out the transition function now.