Taylor expansion of $\frac{1}{z}$ at $z=3$
We first set $w=z-3$
$$\frac{1}{w}=\frac{1}{1--(w-1)}=\frac{1}{1-(-w+1)}=\sum_{n=0}^{\infty}(-w+1)^n=\sum_{n=0}^{\infty}(-1)^n(w-1)^n$$
Plug in $w=z-3$ we get
$$\sum_{n=0}^{\infty}(-1)^n(z-4)^n$$
and $|z-4|< 1$
Is it correct?
Consider this
\begin{eqnarray} \frac{1}{z} &=& \frac{1}{3 - (3 - z)} &=& \frac{1}{3} \frac{1}{1 - (1 - z/3)} \\ &=& \frac{1}{3} \sum_{k=0}^{+\infty} \left(1 - \frac{z}{3}\right)^{k} &=& \sum_{k=0}^{+\infty} (-1)^k\frac{(z - 3)^k}{3^{k+1}} \tag{1} \end{eqnarray}
For $|1 - z/3| < 3$. A fast way for checking whether your approach is working is to plot the first $N$ terms of the series. The plot below shows that: top is your approach, bottom is Eq. (1), $f(x) = 1 / x$
An even simpler way is to evaluate the series at $z=3$, for example, Eq (1) yields
$$ \left.\sum_{k=0}^{+\infty} (-1)^k\frac{(z - 3)^k}{3^{k+1}}\right|_{z=3} = \frac{1}{3} + \left[1 + \left(1 + \frac{z}{3}\right) + \left( 1 - \frac{z}{3}\right)^2 + \cdots\right]_{z= 3} = \frac{1}{3}[1 + 0 + 0 + \cdots] = \frac{1}{3} $$
Your expression, on the other hand yields
$$ \left.\sum_{k = 0}^{+\infty}(-1)^k(z - 4)^k\right|_{z = 3} \sum_{k=0}^{+\infty}(-1)^k (-1)^k = \sum_{k = 0}^{+\infty} 1 $$
which definitely does not converge, that's the reason for the numerical explosion in the top panel of the figure above