Find $\displaystyle \int_0^{\pi/2} f(x)\,dx$ if$$f(x)=\begin{cases} \sin(x), & \text{if } \cos(x)\in \mathbb{Q}\\\sin^2(x), & \text{if } \cos(x) \notin \mathbb{Q} \end{cases}$$
I am not able to solve this problem. This problem is from Real analysis by royden.
On
Theorem : If $g$ is Riemann integrable in $[a,b]$ and $f=g$ almost everywhere in $[a,b]$ then $f$ is Lebesgue integrable and $\displaystyle (L)\int_a^bf(x)\,dx=(L)\int_a^bg(x)\,dx=(R)\int_a^bg(x)\,dx$.
Here , consider $g(x)=\sin^2(x)$. Then , as $m(\mathbb Q\cap [0,\pi/2])=0$ , $f=g$ almost everywhere in $[0,\pi/2]$. So , $$(L)\int_0^{\pi/2}f(x)\,dx=(R)\int_0^{\pi/2}g(x)\,dx=\int_0^{\pi/2}\sin^2 x\,dx=\frac{\pi}{4}.$$
The integral of the function you provide is equivalent to $\int_0^{\pi/2} \sin^2x\,dx$, as Michael Hardy points out in the comments above. The reason for this lies in the difference between being "countably infinite" and "uncountably infinite". This is a very deep topic and is generally studied using set theory. What you need to know is that the cardinality" of a set is the number of elements in the set. When we look at infinite sets some properties of finite sets hold, cardinality among these. The first major works in this field were published by Georg Cantor; in them he showed that the set of rationals has a cardinality of $\aleph_0$ and that the sets of real numbers and irrationals have cardinality $c$ (continuum). Further, Cantor proved that $\aleph_0 < c$ using a clever argument concerning inequalities (although this was proved later than his other results).
A final result key here is that subsets generally inherit the cardinality of the set they are formed from when discussing infinite sets and infinite subsets. For example, the cardinality of the set formed by taking the rationals between $0$ and $1$ is the same as the cardinality of the set on the domain $0$ to $10$ (this results also holds for real numbers and irrational numbers). The term "uncountably infinite" is often used to denote cardinality $c$... this can be intuitively understood by imagining listing off the real numbers between $0$ and $1$. Unfortunately, between any $a$ and $b$ we choose there are infinite reals, so we will never make any progress given any domain. Likewise, we use the term "countably infinite" for cardinality $\aleph_0$. To understand this, imagine listing out the rationals between $0$ to $1$ in the following way: $\frac{1}{2}, \frac{1}{3}, \frac{2}{3}, \frac{1}{4}, \frac{3}{4}, \frac{1}{5}\cdots$ (We are basically listing out all $\frac{p}{q}$ not already in the list for a given $q \geq 2$ and $p$ from $0$ to $q$ for positive, non-zero integers $p$ and $q$.) This algorithm allows us to list the rationals in a given domain (given infinite time); as such, we term this "countably infinite".
In conclusion, let's create two sets, one composed of all rationals and once composed of all real numbers. The former is denoted by $\Bbb Q$ and has cardinality $\aleph_0$. The latter is denoted by $\Bbb R$ and has cardinality $c$. We then see that $\Bbb R\backslash\Bbb Q = \Bbb I$, where $\Bbb I$ denotes the set of irrational numbers. (This is basically saying that the if we take out all the rational numbers from the set of real numbers we get the irrational numbers. This makes sense as any real number is either rational or irrational.) However, Cantor has already shown that $\Bbb I$ has cardinality $c$, so we see that for any domain we will have uncountably infinite irrationals and countably infinite rationals. We thus say that "the measure of $\Bbb Q$ is $0$ within this interval (to help understand this, imagine your friend comes up with a whole number on the domain $(0,\infty)$ and asks you to guess the number. Any number you choose will have an infinitesimal chance of being correct. The percent chance that you are right is less than any real number, so we can, in a sense, say your chances are $0\%$. This is not a perfect analogy, [the event could still happen given my criteria] but I don't claim it to be rigid, just to help you intutively grasp this topic). We can thus see the set of rationals within a given domain is "negligible" compared to the set of irrational numbers within the same domain, eliminating the first condition in your function and yielding the desired result.
Note: here are a list of Wikipedia pages corresponding to some topics in the field of mathematics. These pages might be a little above your current understanding, but should lead you to further sources of information.
Aleph Number ($\aleph_0$)
Cardinality
Cantor's Diagonal Argument
Real Numbers
Rational Numbers
Complement (Set Theory)