Let $F : \mathbb R \rightarrow \mathbb R$ be non-decreasing and right-continuous. Then there exists a unique Borel measure $\mu_F$ on $\mathbb R$ such that for any interval $J \subset \mathbb R$ :
$$\mu_F(J) = \begin{cases} F(t)-F(s) \quad &\text{for } J = (s,t] \\ F(t)-F(s-) \quad &\text{for } J = [s,t] \\ F(t-)-F(s) \quad &\text{for } J = (s,t) \\ F(t-)-F(s-) \quad &\text{for } J = [s,t) \end{cases} $$ This is the standard definition of a Lebesgue Stieltjes measure on $\mathbb R$.
On Wikipedia https://en.wikipedia.org/wiki/Lebesgue%E2%80%93Stieltjes_integration , for $F : [a,b] \rightarrow \mathbb R$ non-decreasing and right-continuous, they define the Stieltjes measure $\hat{\mu}_F$ associated to $F$ as the unique Borel measure on $[a,b]$ satisfying $$\hat{\mu}_F((s,t]) = F(t)-F(s) \ \forall (s,t] \subset [a,b], \quad \hat{\mu}_F(\{a\}) = 0$$
Isn't this definition a bit weird ? Because for $F : \mathbb R \rightarrow \mathbb R$ non-decreasing and right-continuous, if we fix $[a,b] \subset \mathbb R$, the restriction of the measure $\mu_{F}$ to $[a,b]$ does not coincide with the measure $\hat{\mu}_F$ on $[a,b]$ unless $a$ is a continuity point of $F$.
I'm asking this because I'm reading Theory of Ordinary Differential Equations from Coddington and the following "Integration Theorem" is stated :
Let $I = [a,b]$ be a compact interval, let $(F_n)_{n=1}^{\infty}, F : [a,b] \rightarrow \mathbb R$ be uniformly bounded, non-decreasing and right-continuous functions such that for all $\lambda \in [a,b]$ : $$\lim \limits_{n \to +\infty}F_n(\lambda) = F(\lambda)$$ Then for any continuous functions $f \in C^0([a,b])$ : $$\lim \limits_{n \to +\infty} \int_a^b f(\lambda) dF_n(\lambda) = \int_a^b f(\lambda) dF(\lambda)$$
I guess that here, $dF_n$ and $dF$ refers respectively to $\hat{\mu}_{F_n}$ and $\hat{\mu}_F$ (otherwise, the statement does not make sense). But later, they use this theorem when $F_n$ and $F$ are defined everywhere on $\mathbb R$, $\lim \limits_{n \to +\infty} F_n(\lambda) = F(\lambda)$ everywhere on $\mathbb R$, and where $dF_n$ and $dF$ refers respectively to $\mu_{F_n}$ and $\mu_F$ but this does not seem true (take $f \equiv 1$ and $a,b$ not continuity points of $F$).
To answer your question, no it's not weird. :) That is, I don't think the definition is weird. It makes sense that it would not use information about $F$ outside of $[a,b]$. And I agree, the "Integration Theorem" must have in mind $\hat{\mu}_{F_n}$ and $\hat{\mu}_{F}$ since $F_n(a-)$ need not equal or even approach $F(a-)$. To use $\mu_{F}$ and $\mu_{F_n}$ restricted to $[a,b]$ could introduce an arbitrary point mass at $a$ that could violate equality. Perhaps the assumption then becomes $F_n(\lambda)\rightarrow F(\lambda)$ as $n\rightarrow \infty$ for all $-\infty <\lambda <\infty$. You would still have the point mass at $a$ but in the limit the integrals would approach the same value. I'd look closely at the text and see if the definition of $\int_a^b \cdot \;dF$ precludes any point mass at $a$.