According to my calculations $$\sum_{n=1}^{\infty} (-1)^n \zeta(n + 1) = -1$$ but for example WolframAlpha says that the sum doesn't converge, so have I done anything wrong (my calculations down below)?
My calculations:
Start with the taylor series for the digamma function: $$\psi(z) = -\gamma - \sum_{n=1}^{\infty}\zeta(n+1)(1-z)^n$$ Plugging in $2$: $$\psi(2) = -\gamma - \sum_{n=1}^{\infty}\zeta(n+1)(-1)^n$$ But since $\psi(2) = 1 - \gamma$, we have: $$1 - \gamma = -\gamma - \sum_{n=1}^{\infty}\zeta(n+1)(-1)^n$$ And we have arrived at our result that: $$\sum_{n=1}^{\infty} (-1)^n \zeta(n + 1) = -1$$
Recall that $\zeta(s) = \sum_{k=1}^{\infty} \tfrac{1}{k^s}$, so $\lim_{s \to \infty} \zeta(s) = 1$. Thus, the terms of $\sum (-1)^{n} \zeta(n+1)$ don't go to zero, and the sum doesn't converge.
However, there is a sense in which you are right. Let's break up your sum as $$\sum_{n=1}^{\infty} (-1)^n + \sum_{n=1}^{\infty} (-1)^{n} \left( \zeta(n+1)-1 \right).$$
We have $$\sum_{n=1}^{\infty} (-1)^{n} \left( \zeta(n+1)-1 \right) = \sum_{n=1}^{\infty} \sum_{k=2}^{\infty} \frac{(-1)^n}{k^{n+1}}.$$ I claim that this sum is absolutely convergent; to check this, note that $$\sum_{n=1}^{\infty} \sum_{k=2}^{\infty} \frac{1}{k^{n+1}} = \sum_{k=2}^{\infty} \frac{1/k^2}{1-1/k} = \sum_{k=2}^{\infty} \frac{1}{k(k-1)}$$ which converges. Thus, we may rearrange the sum into $$\sum_{k=2}^{\infty} \sum_{n=1}^{\infty} \frac{(-1)^n}{k^{n+1}} = \sum_{k=2}^{\infty} \frac{-1/k^2}{1+1/k} = \sum_{k=2}^{\infty} \frac{-1}{k(k+1)} = \sum_{k=2}^{\infty} \left( \frac{1}{k+1} - \frac{1}{k} \right)= - \frac{1}{2}.$$
The sum $\sum_{n=1}^{\infty} (-1)^n$ does not converge, but it is summable by any of a number of techniques for treating divergent series, such as Abel summation or Cesaro summation. In the sense of those summation methods, we have $\sum_{n=1}^{\infty} (-1)^n = \tfrac{-1}{2}$. So, in the sense of those summation methods, your sum is $\tfrac{-1}{2} + \tfrac{-1}{2} = -1$.