I struggle to understand the concept of quadratic convergence. It is said that it implies linear convergence but let's say we have:
$$|x_{n+1}-x_*|<C|x_{n}-x_*|^2$$
and $$C=1$$
If $ |x_{n}-x_*|$ = 2 , then $|x_{n+1}-x_*|\leq 4$ and we get reclusively $$\forall k\in N ,|x_{n+1+k}-x_*|<4^{2*k}$$
Which does not seem to imply something good for our error.
We do have the assumption that $x_n \to x_*$ then at some point $|x_{n+1+k}-x_*|<1$ but still it does not seem good considering the boundness we got before this happening. How could you assume it does not take centuries to get to $|x_{n+1+k}-x_*|<1 $
I am probably missing something here
There is no misunderstanding. When we are discussing the order of convergence, the baseline assumption is that the sequence converges... So, as you say, $|x_n-x^*| < 1$, for large enough $n$.
In the situation you describe, you could control $|x_n-x^*|$ using other estimates, until the quadratic inequality becomes effective. For instance $$ |x_n-x^*|\leq \frac{L}{1-L}|x_n-x_{n-1}|, $$
where $L$ is the Lipschitz constant for the iterating function.