Let's consider a function $f(x)$ where $x \in \mathcal X$, and $f(x)$ is sampled from a Gaussian Process. Also assume that for each $x\in\mathcal X$, $f(x)$ has a fixed unknown mean which is $\mu(x)$.
Question: Can we say anything about the relationship between $\mu_t(x)$ and $\mu(x)$? I mean is there any interval around $\mu_t(x)$ (for example $[\mu_t(x) - \alpha\sigma_t(x),\mu_t(x) + \alpha\sigma_t(x)]$) such that $\mu(x)$ lies in it with high probability?
Edit: $t$ is indicating the iterations here, i.e. at time $t$ after getting a sample from $f$ we update all the means and variances (having a special kernel), and same for $t+1$ and so on.