Consider a linear trend model, i.e. $$X_t = \mu + \beta\cdot t + N_t$$ We then have $$(\nabla X)_t = (1 - B)X_t = \beta + (1 - B)N_t$$ where $\nabla$ is the lag-1 difference operator and $B$ is the backshift operator. In the lecture notes at my university, it is stated that
If differenced noise $(1 - B)N_t$ is stationary, we can estimate slope $\beta$ from data as the mean of the differenced time-series $\nabla X$.
Note that $$\mathbb{E}[(\nabla X)_t] = \mathbb{E}[(1 - B)X_t] = \beta + \mathbb{E}[(1 - B)N_t]$$ I am wondering, if $\mathbb{E}[(\nabla X)_t]$ is still a good estimate of $\beta$ even if $\mathbb{E}[(1 - B)N_t]\neq 0$?
Also, why do we assume that $(1 - B)N_t$ is stationary in the quoted statement above? Are there any useful implications of this assumptions?
Example
Consider the time series \begin{equation} X_t = \mu + \beta\cdot t + N_t \end{equation} where $\mu,\beta\in\mathbb{R}$ and the noise is modeled as Gaussian random walk with drift, that is, \begin{equation} N_t - N_{t-1}\sim\mathcal{N}(\kappa,\sigma^2) \end{equation} for some $\kappa\neq 0$ and $\sigma^2 > 0$. In this case, we obtain \begin{align} \mathbb{E}[(\nabla X)_t] &= \mathbb{E}[(1 - B)X_t]\\ &= \beta + \mathbb{E}[(1 - B)N_t]\\ &= \beta + \kappa \end{align} If $\kappa$ is large in magnitude, then $\mathbb{E}[(\nabla X)_t]$ is not a particularly good estimate of $\beta$.
If you want $\mathbb{E}[\nabla X]$ to be a good estimator for the slope, then yes, the closer $\mathbb{E}[(1-B)N_t]$ is to $0$, the better the estimation will be, but that's slightly besides the point.
Say that, as you wrote, $\mathbb{E}[(1-B)N_t] = \kappa$, where $\kappa$ is known. Then we simply take $\mathbb{E}[\nabla X] - \kappa$ to be the estimator. On the other hand, if $\kappa$ is not known, $\mathbb{E}[\nabla X]$ will be an unreliable estimator, because we don't know how to split it into $\beta$ and $\kappa$. It's not like we are emotionally attached to this particular estimator, it's just that it is the key concept for solving such problems, and simple modifications to the problem that can be solved very easily are not mentioned, because there are too many modifications to mention.
Hope this provided a useful change in perspective.