Why is it that every linear operator $f: \mathbb R^n \to \mathbb R^m$ is bounded and therefore continuous, but why is it that $A:X \to Y$ between two normed spaces is continuous iff bounded? That means that $A:X\to Y$ is not necessarily bounded but $f:\mathbb R^n\to \mathbb R^m$ is always bounded?
Also I have trouble understanding the $\epsilon$-$\delta$ definition to show that it is bounded and therefore continuous. Here we define the application $f(x) = Ax$ where $A$ is a $m\times n$ matrix. Then by Cauchy Schwarz we show that $\|Ax\|_2 \le C\|x\|_2$ and this part I dont get.
They choose delta such that $\delta = C/\epsilon$.
I am really bad at knowing what do define $\delta$ as and I dont get how this choice shows continuiuty. I feel like it shows Lipschitz.
For linear operators, it's always true that boundedness and continuity are equivalent. There's no difference in that regard between finite/infinite dimensions. The weird/nice thing about finite dimensions is this: since there are only finitely many directions to stretch, a linear map between finite dimensional spaces is forced to be bounded (and therefore continuous).
If I have a finite dimensional space $X$ with orthonormal basis $\{e_1, ..., e_n\}$, and a linear map $A: X \rightarrow Y$, then each of our basis vectors is stretched by some amount by $A$ and then stuck into $Y$, i.e. $\|Ae_i\| = c_i\|e_i\| = c_i$ for some $c_i \ge 0$. Now we want to show that the $c_i$ "stretching factors" control the stretching for all vectors.
Let $C = \sqrt{\sum_{i=1}^n c_i^2}$. We analyze how much a generic element of $X$ can be stretched by the map $A$. Let $x = \sum_i a_i e_i$. Then we have:
$$\begin{align} \|Ax\| &= \|\sum a_i Ae_i\| \\ & \le \sum \|a_iAe_i\| \\ &\le \sum |a_i|\|Ae_i\| = \sum |a_i|c_i = \langle \sum |a_i|e_i, \sum c_i e_i\rangle \\ &\le \|\sum |a_i|e_i\|\|\sum c_i e_i\| = C\|x\|. \end{align}$$
So the most stretching that $A$ can do is by a factor of $C$. Therefore (by the above) $A$ is bounded (equivalently, continuous).
But, like Hagen said, weird things can happen in infinite dimensions. If I have an infinite basis $\{e_i\}_{i=1}^\infty$, then I can make a linear map that stretches each basis vector $e_n$ by a factor of $n$. The resulting map won't be bounded (since the $n$ go on forever), and therefore it won't be continuous. There's just too much room for stretching in infinite dimensions.
About your second question: you're right that we've shown that $A$ is Lipschitz. But this implies (uniform) continuity. Note that we have $\|Ax\| \le C\|x\|$, so $\|Ax - Ay\| = \|A(x-y)\| \le C\|x-y\|$. Now we see that by controlling the distance from $x$ to $y$, we can control the distance from $Ax$ to $Ay$. In particular, if we have an $\epsilon>0$ and we want to make sure that $\|Ax - Ay\|<\epsilon$, then it's enough to make sure that $\|x-y\|<\frac{\epsilon}{C}$, so we choose $\epsilon/C$ as our $\delta$.