A Gaussian process $z(s)$ can be established by convolving a gaussian white noise process $x(s)$ with a smoothing kernel $k(s)$ http://ftp.stat.duke.edu/WorkingPapers/01-03.pdf
$$z(s)=\int_{S}^{} \! k(u-s) x(u).du \ \ \text{where } s\in R $$
As a result, the covariance between any two points $s$ and $s'$ can be expressed as follows
$$ c(s, s')=\int_{S}^{} \! k(u-s) k(u-s').du= \int_{S}^{} \! k(u-(s-s')) k(u).du$$
I need help understanding two issues :
1) white noise process is discontinuous and thus Riemann integration cannot be used in the first equation
2) How is the covariance equation derived
3) How did $s'$ jump to the first kernel
White noise can only be defined in the sense of distributions or as a measure. A good definition can be found in Adler and Taylor (2007, Sec. 1.4.3), see also this SE answer.
To calculate second moments you want to use stochastic integration Adler and Taylor (2007, sec. 5.2) (also see below) for deterministic functions $f,g$ $$ \mathbb{E}[W(f)W(g)] \overset{\text{def.}}=\mathbb{E}\Bigl[\Bigl(\int f(x)W(dx)\Bigr)\Bigl(\int g(x)W(dx)\Bigr) \Bigr] =\int f(x)g(x) dx, \tag{1} $$ which can be viewed as a special case of the Itô Isometry.
Convolution
We can consider convolutions as a special case $$ (f*W)(t) = \int f(t-s)W(ds) = W(f(t-\cdot)) $$ then the covariance function (expectation is zero) is given by $$ C(t,s) = \mathbb{E}[ (f*W)(t)(f*W)(s)] = \int f(t-x)f(s-x)dx $$
Stochastic Integration
The trick to prove (1), is to show that the mapping $$ W:\begin{cases} L^2(\mathbb{R}^n, \mathcal{B}, \nu) &\to L^2(\Omega, \mathcal{A}, \mathbb{P})\\ f &\mapsto W(f) := \int f(t) W(dt) \end{cases} $$ preserves the scalar product. We first consider simple functions $f=\sum_{i=1}^n a_i \mathbf{1}_{A_i}$ for disjoint $A_i$, then $$ W(f)=\int f(t) W(dt) \overset{\text{def.}}= \sum_{i=1}^n a_i W(A_i) $$
To calculate the scalar product between $f$ and $g=\sum_{i=1}^n b_i \mathbf{1}_{B_i}$ we assume without loss of generality $A_i=B_i$ (consider all of their interesections). Then $$\begin{aligned} \langle W(f), W(g) \rangle_{L^2(\mathbb{P})} &= \mathbb{E}\Bigl[\sum_{i=1}^n a_i W(A_i) \sum_{j=1}^n a_j W(A_j)\Bigr]\\ &= \sum_{i=1}^n a_i b_i \mathbb{E}[W(A_i)^2]\\ &= \int f(t) g(t) \nu(dt)\\ &= \langle f, g\rangle_{L^2(\nu)} \end{aligned}$$ Since the simple function are dense in $L^2$ and the scalar product is continuous we can deduce that $W$ is an isometry (where $W(f)$ for general $f$ is defined as the limit of $W(f_n)$ for simple functions $f_n$ approximating $f$). (1) then directly follows from the respective definitions of the scalar product (i.e. the second and penultimate term).