Suppose that $X_t$ is an Itô process: $$ \mathrm{d} X_t = \mu_t \mathrm{d} t + \sigma_t \mathrm{d} W_t $$ where $\mu_t$ and $\sigma_t$ are predictable processes wrt. $\mathcal{F}_t$. Then we have \begin{align} \lim_{\Delta \to 0} \frac{1}{\Delta t} \mathbb{E} \left[ X_{t+\Delta t} - X_t | \mathcal{F}_t \right] &= \mu_t \\ \lim_{\Delta \to 0} \frac{1}{\Delta t} \operatorname{Var} \left[ X_{t+\Delta t} - X_t | \mathcal{F}_t \right] &= \sigma_t^2 \end{align}
This is seems to be true, and the Markovian case where $\mu_t = \mu(t, X_t), \sigma_t = \sigma(t, X_t)$ sort of appears in Pavliotis' Stochastic Processes and Applications.
I'm looking for a reference for this general case.
I am unsure about the completely general case, i.e. $\mu, \sigma$ just predictable with no other assumptions, but under some assumptions that I will list below, for almost all $t\geq 0$ in some interval the limits hold almost surely.
Let $\left(\Omega, \mathscr{F}, \left\{\mathscr{F}_t\right\}, \mathbb{P}\right)$ be filtered probability space such that the filtration satifies the usual conditions. Let $\left(W\right)_{t\geq 0}$ be a Brownian motion defined on this space and $\left(X\right)_{t\geq 0}$ an Itô process defined by $$dX_t = \mu_t dt + \sigma_t dW_t.$$ We suppose that $\left(\mu_t\right)_{t \geq 0}$ and $\left(\sigma_t\right)_{t\geq 0}$ are $\mathscr{F}_t$-adapted and almost surely left-continuous, which makes the two processes predictable. Furthermore, we suppose that there exists a $T_{max}\in \left(0, \infty\right]$ such that for all $T < T_{max}$, $$ \mathbb{E}\left[\sup_{t \leq T}\left|\mu_t\right| + \sup_{t \leq T}\left|\sigma_t\right|^2\right] < \infty. $$ Note that this implies that $\left(\int_0^{t \wedge T}\sigma_s dW_s\right)_{t\geq 0}$ is a martingale, in fact a UI martingale bounded in $L^2$.
We fix a $T < T_{max}$.
First, we identify the almost sure limit of the quanties inside the mean and variance. For almost any $\omega \in \Omega$, we have by Lebesgue's differentiation theorem that for almost all $t \in \left[0,T\right]$ $$\lim_{\Delta t \downarrow 0} \left(\frac{1}{\Delta t}\int_t^{t + \Delta t}\mu_s\left(\omega\right)ds\right) = \mu_t\left(\omega\right) \qquad \lim_{\Delta t \downarrow 0} \left(\frac{1}{\Delta t}\int_t^{t + \Delta t}\sigma^2_s\left(\omega\right)ds\right) = \sigma^2_t\left(\omega\right),$$ and if we replace $t$ with $t+$ in what is above, i.e. $\mu_{t+}\left(\omega\right) = \lim_{s \downarrow \downarrow t} \mu_s\left(\omega\right)$, these limits hold for all $t \in \left[0,T\right]$.
Next for $t < T$ and $\Delta t > 0$ sufficiently small, we compute the conditional mean and variance of the increments. $$ \mathbb{E}\left[\left. X_{t + \Delta t} - X_t\right|\mathscr{F}_t\right] = \mathbb{E}\left[\left. \int_t^{t + \Delta t}\mu_s ds + \int_t^{t + \Delta t}\sigma_s dW_s\right|\mathscr{F}_t\right] = \mathbb{E}\left[\left. \int_t^{t + \Delta t}\mu_s ds \right|\mathscr{F}_t\right],$$ the second term in the expectation being zero as the stochastic integral is a martingale. $$ \text{Var}\left[\left. X_{t + \Delta t} - X_t \right|\mathscr{F}_t\right] = \mathbb{E}\left[\left. \left(X_{t + \Delta t} - X_t -\int_t^{t + \Delta t}\mu_s ds\right)^2\right|\mathscr{F}_t\right] = \mathbb{E}\left[\left. \left(\int_t^{t + \Delta t}\mu_s ds + \int_t^{t + \Delta t}\sigma_s dW_s -\int_t^{t + \Delta t}\mu_s ds\right)^2\right|\mathscr{F}_t\right] = \mathbb{E}\left[\left. \left(\int_t^{t + \Delta t}\sigma_s dW_s\right)^2\right|\mathscr{F}_t\right] = \mathbb{E}\left[\left. \int_t^{t + \Delta t}\sigma^2_s ds\right|\mathscr{F}_t\right],$$ where the last equality comes from a conditional version of Itô's isometry.
Finally for $\Delta t > 0$ sufficiently small, we have $$ \left|\frac{\int_t^{t + \Delta t}\mu_s ds}{\Delta t}\right| \leq \sup_{t \leq T}\left|\mu_t\right| \quad \text{ and } \quad \left|\frac{\int_t^{t + \Delta t}\sigma^2_s ds}{\Delta t}\right| \leq \sup_{t \leq T}\left|\sigma_t\right|^2,$$ thus by our hypotheses and Lebesgue's dominated convergence theorem, $$ \lim_{\Delta t \downarrow 0}\mathbb{E}\left[\left.\frac{1}{\Delta t}\int_t^{t + \Delta t}\mu_s ds\right| \mathscr{F}_t\right] = \mathbb{E}\left[\mu_{t+} \left| \mathscr{F}_t \right. \right] = \mu_{t+},$$ and $$ \lim_{\Delta t \downarrow 0}\mathbb{E}\left[\left.\frac{1}{\Delta t}\int_t^{t + \Delta t}\sigma^2_s ds\right| \mathscr{F}_t\right] = \mathbb{E}\left[\sigma^2_{t+} \left| \mathscr{F}_t \right. \right] = \sigma^2_{t+}.$$ Since we are working with conditional expectations, these inequalities hold almost surely.
For almost all $t \in \left[0, T\right]$, $\mu_{t+} = \mu_t$ and $\sigma_{t+} = \sigma_t$, again almost surely. To see this consider $$ \mathbb{E}\left[\int_0^T \left|\mu_t - \mu_{t+}\right|dt\right].$$ As for almost every $\omega \in \Omega$ $\mu\left(\omega\right)$ is almost surely left continuous and bounded on $\left[0, T\right]$, the integrand above is non-zero for only for countably many $t$. Thus after interchanging the expectation and integral by Fubini, for almost all $t \in \left[0, T\right]$, we have that $ \mathbb{E}\left[\left|\mu_t - \mu_{t+}\right|\right] = 0 $ and $\mu_t = \mu_{t+}$ almost surely. A similar argument applies for $\sigma_{t+}$.
In conclusion, given our two hypotheses for almost all $t \in \left[0,T\right]$, we have that \begin{align} \lim_{\Delta t \to 0} \frac{1}{\Delta t} \mathbb{E} \left[ X_{t+\Delta t} - X_t | \mathscr{F}_t \right] &= \mu_t \\ \lim_{\Delta t\to 0} \frac{1}{\Delta t} \operatorname{Var} \left[ X_{t+\Delta t} - X_t | \mathscr{F}_t \right] &= \sigma_t^2 \end{align} almost surely. As final note, I would like to highlight that the convergence is for a fixed $t$ and in general we may not be able to interchange the almost all $t$ and almost surely.