Performing long division, I calculated the $2$-adic inverse of $3$ to be $$1-2+4-8+...$$
Then I noticed that I could get the same result in a more sleek way by noting that $$\frac 1 3 = \frac {1}{1-(-2)}$$
and applying the geometric series formula which works because $||-2||_2=2^{-v_p(-2)}=1/2<1$. But then I also noticed that
$$\frac 1 3 = \frac {-1}{1-4}=-[1+4+16+64+...]$$
doesn't work. Why?
I don’t know what you mean when you say “it doesn’t work”. Your computations are both correct. In standard expansion, where the coefficients of the powers of $2$ are only zero and one, we also have $1/3=1+2+8+32+128+\cdots$, in other words, $1/3$ is equal to $1+\sum_{j=0}^\infty2^{2j+1}$. It’s the same as it is for real numbers, where any real has many different representations as the sum of a convergent series.
EDIT: In response to @Rodrigo’s question of how I found this, let me say: I believe firmly in writing $p$-adic numbers in $p$-ary notation, but extending (potentially) infinitely to the left, as opposed to real numbers, written in notation with respect to any integer radix $n>1$, extending infinitely to the right. So, the binary expansion of $-1$ is $\cdots111111;$ You find this just by doing a standard subtraction, take $0$ and subtract $1$. The first digit will be $1$ with a borrow, forcing subtraction again of $1$ from zero, etc. Now, to divide $3$ into $1$, you set up your division a little differently from standard division, you work right to left. Your first digit will be $1$, and when you multiply $3=11;$ times this, you must subtract $11;$ from $\cdots0001;$ The difference is $\cdots11110;$, which tells you that the next digit of the quotient is $1$. Subtract $110;$ from $\cdots111110;$ and get $\cdots1111000;$ This tells you that in order, the next two digits of the quotient are zero and $1$. Now you see that you’re in a repetitive pattern, so that the full expansion of $1/3$ is $\cdots01010101011;$ Remember that when you do computations like this, your carries proceed to the left, exactly as in ordinary elementary-school computation.
If you’re having trouble absorbing this, make sure you understand the expansion of $-1$, and (of course) check that $-1=\sum_{j=0}^\infty2^j$.