In the book of Mathematical Analysis II by Zorich, at page 135, it is asked that
Show by example that if the values of the function $F(x)$ that occurs in Fubini’s theorem, which in the theorem were subjected to the conditions $\underline J (x) ≤ F(x) ≤ \bar J(x)$ at all points where $\underline J(x) < \bar J(x)$, are simply set equal to zero at those points, the resulting function may turn out to be nonintegrable. (Consider, for example, the function $f(x,y)$ on $\mathbb{R}^2$ equal to $1$ if the point $(x,y)$ is not rational and to $1 − 1/q$ at the point $(p/q, m/n)$, both fractions being in lowest terms.)
I couldn't understand what the author is asking. I mean what does he mean by "set equal to zero" ? Plus, what kind of grammatical structure does the author using in "[...] at those points, the resulting function may turn out to be nonintegrable. [...]"
Reference:
and $$F(x) = \int_y f(x,y) dy$$
I took a look at the book by Zorich, to confirm that these are Riemann integrals under consideration and to clarify the notation as ,
$$F(x) = \int_0^1 f(x,y) \, dy, \\ \overline{J}(x) = \overline{\int}_0^1 f(x,y) \,dy, \\ \underline{J}(x) = \underline{\int}_0^1 f(x,y) \,dy,$$
with $\overline{J}(x)$ and $\underline{J}(x)$ denoting upper and lower Darboux integrals, which always exist when $f$ is bounded.
The question is poorly worded. At points where $\underline{J}(x) < \overline{J}(x)$ with strict inequality, the Riemann integral $F(x)$ does not exist, and it is meaningless to write $\underline{J}(x) \leqslant F(x) \leqslant \overline{J}(x)$. If $F(x)$ exists then of course $\underline{J}(x) = F(x) = \overline{J}(x)$.
What is really being asked is to produce an example where $F(x)$ exists at points where $\underline{J}(x) = \overline{J}(x)$, $F(x)$ fails to exist at points where $\underline{J}(x) < \overline{J}(x)$ and is defined to be $F(x) = 0$, yet $F$ is not Riemann integrable over $[0,1]$.
With the suggested function $f$, if $x \in[0,1]$ is irrational then $f(x,y) =1 $ for all $y \in [0,1]$. With $x$ thus fixed, the function $y \mapsto f(x,y)$ is Riemann integrable over $[0,1]$ with $$\overline{J}(x) = \underline{J}(x) = F(x) = \int_0^1 f(x,y) \, dy = 1.$$
So, in this case, nothing unusual happens.
On the other hand, if $x \in (0,1]$ is rational then it is of the form $p/q$ in lowest terms and
$$f(p/q,y) = \begin{cases}1, \qquad\qquad\qquad \text{if } y \in [0,1] \text{ is irrational} \\ 1 - 1/q < 1, \quad \text{ if } y \in [0,1] \text{ is rational}\end{cases}.$$
This is a Dirichlet-type function which is not Riemann integrable and $F(x)$ is undefined. However, the upper and lower integrtals are, respectively, $\overline{J}(p/q) = 1$ and $\underline{J}(p/q) = 1- 1/q.$
Hence, if $x \in (0,1]$ is rational then we have the case mentioned in the question where
$$\tag{*}\underline{J}(x) < \overline{J}(x)$$.
By defining $F(x) = 0$ at points where (*) holds, we now have a function where $F(x) = 1$ if $x$ is irrational and $F(x) = 0$ when $x \in (0,1]$ is rational. Regardless of how $F(0)$ is defined, this Dirichlet function is not Riemann integrable.