Suppose we have a function defined on an interval $[a,b]$. Let $R$ be the Riemann integral, and let $M$ be the limit as $n\rightarrow\infty$ of the Riemann sum in which the interval is partitioned into $n$ equal subintervals and the function is sampled at the center of each subinterval.
What are the least pathological functions for which $R$ differs from $M$? Differing could mean that they're both defined but they're unequal, or it could mean that one is defined and the other is not. "Pathological" is of course in the eye of the beholder, but basically I'm looking for something that would be more likely to be the kind of example that would actually have some application in physics, economics, engineering, etc. For example, I can imagine modeling the stock market using a very badly behaved function. Maybe Brownian motion...?
The least pathological example I can think of is pretty pathological. Let $f(x)$ be 1 if $x$ is rational, and 0 if $x$ is irrational. Then on the interval $[0,1]$, $R$ is undefined but $M$=1. I don't hold it against this function too much that it's discontinuous everywhere -- I can imagine my stock market or Brownian motion functions doing that. But the particular way in which it's discontinuous is artificially cooked up in a way that I don't believe would occur in real applications.
A function $f:[a,b]\to\mathbb R$ is Riemann integrable ($f\in\mathcal R[a,b]$) iff there is a number $R$ such that for any $\epsilon>0$ there is a $\delta>0$ such that for any partition $P=\{a=x_0<x_1<\dots<x_n=b\}$ and any tags $x_0^*,\dots,x_{n-1}^*$ with $x_i^*\in[x_i,x_{i+1}]$ for all $i$, we have that if $x_{i+1}-x_i<\delta$ for all $i$, then $|R(f,P,\vec {x_i^*})-R|<\epsilon$, where $$ R(f,P,\vec{x_i^*})=R(f,P,x_0^*,\dots,x_{n-1}^*)=\sum_{i=0}^{n-1}f(x_i^*)(x_{i+1}-x_i). $$
If this is the case, then, in particular, letting $P_n$ be the partition into $n$ intervals of equal size, and letting the tags $y_0^*,\dots,y_{n-1}^*$ be the midpoints of these intervals, we have that $R(f,P_n,\vec{y_i^*})\to R$ as $n\to\infty$. That is, if $f$ is Riemann integrable (that is, the number you call $R$ exists), then the number you call $M$ also exists, and they are equal.
It follows that the only way to have disagreement between $R$ and $M$ is that $M$ is defined but $R$ is not. The Lebesgue criterion for Riemann integrability tells us that $f\in\mathcal R[a,b]$ iff $f$ is bounded, and its set of discontinuities has (Lebesgue) measure zero. Thus in order for $R$ not to be defined, it should be that either $f$ is unbounded, or else that $f$ is "highly discontinuous", that is, its set of discontinuities should have positive outer measure.
Now, that either case of the failure of integrability happens is not enough, because we also need that $M$ is defined. A typical representative of the simplest example I can think of of this situation is the following:
Let $[a,b]=[-1,1]$, and set $$f(x)=\left\{\begin{array}{cl}1/x&\mbox{ if }x\in(0,1],\\ -1/x&\mbox{ if }x\in[-1,0),\mbox{ and }\\ 0&\mbox{ if }x=0.\end{array}\right.$$ The function $f$ is continuous, except at $0$. Since $f$ is unbounded, it is not Riemann integrable (and it does not matter, but the improper integral does not exist either in this example). However, by symmetry, $R(f,P_n,\vec{y_i^*})=0$ for all $n$, so $M=0$.
In this example, $f$ is unbounded in both directions, that is, $\sup_x f(x)=+\infty$ and $\inf_x f(x)=-\infty$. This is essential if we want such a simple example. Indeed, if $f$ is unbounded, but only (say) $\sup_x f(x)=+\infty$, then it is not too hard to see that $f$ must have (at least) countably many discontinuities: If $f$ has only finitely many discontinuities, $\sup_x f(x)=+\infty$ and $\inf_xf(x)>-\infty$, then one can argue that $M$ does not exist, because the sums $R(f,P_n,\vec{y_i^*})$ grow unboundedly.
A simple example with a function of this kind is achieved by letting $[a,b]=[0,1]$, fixing a strictly increasing sequence $t_n$ of irrationals in $[0,1]$ converging to $1$, and letting $f(x)=0$ except at the $t_n$, where we have $f(t_n)=n$. Here $R$ does not exist, by unboundedness, but $M=0$. In this example, $f$ is only discontinuous at $1$ and at the $t_n$.
As you point out, Dirichlet function (given by $f(x)=1$ if $x$ is rational, and $f(x)=0$ otherwise) is a simple example of a bounded function with $R$ undefined and $M=0$, whose set of discontinuities is not of measure zero.