The Wiki page of the Riemann Zeta function contains a series representation for $\zeta(s)$ involving the rising factorial. This representation could also be written as:
$$\zeta(s)= \frac{s}{s-1}-\frac{1}{\Gamma(s)}\sum_{n=1}^\infty \frac{\Gamma(n+s)}{\Gamma(n+2)} \big(\zeta(n+s)-1\big) \qquad s \in \mathbb{C}, s \ne 1 \tag{1}$$
and surprisingly $\Re(\zeta(n+s))>1$ for values of $s$ in the critical strip. So, we might even replace it by its infinite series or its Euler product. Assuming the RH, this would imply all information about the non-trivial zeros would be encoded somehow on the lines $\Re(s) =\frac32,\Re(s)=\frac52,...$
Splitting the series gives:
$$\zeta(s)= \frac{s}{s-1}-\frac{1}{\Gamma(s)}\sum_{n=1}^\infty \frac{\Gamma(n+s)}{\Gamma(n+2)}\zeta(n+s)+\frac{1}{\Gamma(s)}\sum_{n=1}^\infty \frac{\Gamma(n+s)}{\Gamma(n+2)} \qquad s \in \mathbb{C}, s \ne 1 \tag{2} $$
Numerical evidence suggest that $\displaystyle\frac{1}{\Gamma(s)}\sum_{n=1}^\infty \frac{\Gamma(n+s)}{\Gamma(n+2)} = -\frac{s}{s-1}$ for $0 < \Re(s) < 1$, hence we would get:
$$\zeta(s)= -\frac{1}{\Gamma(s)}\sum_{n=1}^\infty \frac{\Gamma(n+s)}{\Gamma(n+2)} \zeta(n+s) \qquad s \in \mathbb{C}, 0 < \Re(s) < 1 \tag{3}$$
Is the latter indeed true for the critical strip? It appears so for $\Im(s)$ small, however for larger $\Im(s)$ computations seem to diverge.
EDIT:
Found a clue, that I share as additional info. Using: $\zeta(s)\Gamma(s) = \int_0^\infty \frac{x^{s-1}}{\mathrm{e}^x-1} \mathrm{d}x, \Re(s) > 1$ we get:
$$\zeta(s)= -\frac{1}{\Gamma(s)}\sum_{n=1}^\infty \left( \frac{1}{\Gamma(n+2)} \int_0^\infty \frac{x^{n+s-1}}{\mathrm{e}^x-1} \mathrm{d}x\right) \qquad s \in \mathbb{C}, 0 < \Re(s) < 1 \tag{4}$$
Swapping the integral and the summation and then evaluating the sum, yields:
$$\zeta(s)= \frac{1}{\Gamma(s)} \int_0^\infty \left(\frac{1}{\mathrm{e}^x-1} -\frac{1}{x} \right) x^{s-1} \mathrm{d}x \qquad s \in \mathbb{C}, 0 < \Re(s) < 1 \tag{5}$$
And (5) already appears in Titchmarsh book on the Riemann Zeta function:
