The context: p-adic convergence
I first ran into the formula:
$$ S = \lim_{n \rightarrow \infty} \sum_{i=1}^n 4 \cdot 5^i - 5^{n+1} = -1 $$
in the context of $p-$adics, where we define a new norm, where for $x = 5^a \cdot b$, $5 \not \lvert b$, the norm is defined as $||x|| \equiv 5^{-a}$. That is, $a$ is the largest power of $5$ in the prime factorization of $x$.
Under this definition, we can see that $\lim_{n\rightarrow \infty}||5^{n+1}|| = 0$, and $\lim_{n\rightarrow \infty} \sum_{i=1}^n 4 \cdot 5^n = \frac{4}{1-5} = -1$. The GP formula is legal since $||5|| = 5^{-1} < 1$.
Hence, the total limit is $-1 - 0 = -1$.
Using the same formula under the usual norm
However, let's now decide to use the $\epsilon-\delta$ definition of convergence of a series, under the usual norm $|x| = abs(x)$ (the absolute value). Now, we claim that the limit of the series $S$ is $-1$. That is, we need to show that
$$ A_n \equiv \sum_{i=1}^n 4 \cdot 5^i - 5^{n+1} \\ \forall \epsilon > 0, \exists N \in \mathbb{N}, \forall n \geq N, |A_n - (-1)|< \epsilon $$
Let's evaluate $|A_n - (-1)|$:
\begin{align*} |A_n - (-1)| &= |A_n + 1| = |4 + 4\cdot5 + 4 \cdot 5^2 + \dots + 4 \cdot 5^n - 5^{n+1} + 1| \\ &= |5 + 4 \cdot 5 + 4 \cdot5^2 + \cdots 4\cdot 5^n - 5^{n+1}| \\ &= |5\cdot 5+ 4\cdot 5^2 + \cdots + 4\cdot 5^n - 5^{n+1}| \\ &= |5^2 + 4\cdot 5^2 + \cdots + 4\cdot 5^n - 5^{n+1}| \\ &= |5\cdot 5^2 + \cdots + 4\cdot 5^n - 5^{n+1}| \\ &= \cdots \\ &= |5\cdot 5^n - 5^{n+1}| \\ &= |0| = 0 \end{align*}
That seems to imply that the series:
$$ S = \lim_{n \rightarrow \infty} \sum_{i=1}^n 4 \cdot 5^i - 5^{n+1} = -1 $$
under the usual norm! But this makes no sense, the terms keep getting bigger, there is no reason this should converge? What am I missing?
If it does indeed converge, then why do we need the $p-$adic norm to explain this situation?
I think you're conflating a couple of things here. Let $S_n$ be the sequence of partial sums of the geometric series, $S_n=\sum_{i=0}^n4\cdot 5^i$. Then it is true for all $n$ that $S_n=5^{n+1}-1$ by the usual formula, so that if we look at the 'associated' sequence $\mathfrak{S}_n=S_n-5^{n+1}$, then $\mathfrak{S}_n$ is $-1$ for all $n$, so $\lim\limits_{n\to\infty}\mathfrak{S}_n = -1$ because it's the limit of a constant sequence.
But what makes the discussion interesting in the context of the $5$-adics isn't that this limit of $\mathfrak{S}_n$ converges; rather, by the explicit formula we have $|S_n-(-1)|_5$ = $|5^{n+1}|_5$ $=5^{-(n+1)}$. And since the norm of the difference between $S_n$ and $-1$ converges to zero as $n\to\infty$, the limit of the $S_n$ — that is, the limit of the sum itself — exists in the $5$-adic topology, and there $\lim\limits_{n\to\infty}S_n=-1$, so we can assign a meaningful value to the sum $\sum_{i=0}^\infty 4\cdot 5^i$.