We are covering Measure Theory and the Borel Algebra $B(\mathbb{R})$ and I am trying to find a nontrivial example of a Borel Set. With the notions of $F_\sigma$ and $G_\delta$ sets, I am wondering if we can find a borel set which is not the countable union or countable intersection of $F_\sigma$ or $G_\delta$ sets.
Mainly, I want to know how "bad" can a set in $B(\mathbb{R})$ be?
On
As an addendum to Noah Schweber's answer, here's a concrete example of a set that is in both $\Sigma_3^0$ and $\Pi_3^0$ (and hence in $\Delta_3^0$) but neither in $\Sigma_2^0$ nor in $\Pi_2^0$. The set of all real numbers in $[0,1]$ whose base-$10$ representations contain (a) infinitely many $1$'s and (b) finitely many digits that are not in $\{1,2\}$.
Requirement (a) gives a set in $\Pi_2^0$ that is not in $\Sigma_2^0$, while requirement (b) gives a set in $\Sigma_2^0$ that is not in $\Pi_2^0$. Finite unions and intersections of sets in $\Sigma_n^0$ and $\Pi_n^0$ produce sets that are provably in $\Delta_{n+1}^0 = \Sigma_{n+1}^0 \cap \Pi_{n+1}^0$.
The relevant topic here is descriptive set theory. The standard texts on the subject are Moschovakis and Kechris; I tend to prefer the latter, especially as a first introduction unless you're already dead-set on becoming a logician, but the former is freely available on the author's website.
The class of Borel sets is vastly more complicated than that. The key point is the Borel hierarchy (and the fact that it doesn't collapse).
The finite levels of the Borel hierarchy are more or less what one would expect:
A set is $\Sigma^0_1$ iff it is open, and a set is $\Pi^0_1$ iff it is closed.
A set is $\Sigma^0_{n+1}$ iff it is the union of countably many $\Pi^0_n$ sets, and a set is $\Pi^0_{n+1}$ iff it is the intersection of countably many $\Sigma^0_n$ sets.
At this point it's a good exercise to check that a set is $\Sigma^0_n$ iff its complement is $\Pi^0_n$.
This notation extends the "F/G" notation you're used to - for example, the $F_\sigma$ sets are exactly the $\Sigma^0_2$ sets, and the $G_\delta$ sets are exactly the $\Pi^0_2$ sets. As we go further you'll see why this notation is much better.
It turns out that this hierarchy is nontrivial:
This isn't easy to prove, though, and relies on the notion of a universal set for these complexity classes (see e.g. this old answer of mine for a summary of this notion).
But it gets worse - there are Borel sets which are not $\Sigma^0_n$ or $\Pi^0_n$ for any $n\in\mathbb{N}$! This follows quickly from $(*)$. Basically, for each $n\in\mathbb{N}$ let $A_n$ be a $\Sigma^0_{n+1}$ set which is not $\Sigma^0_n$ with $A_n\subseteq [2n, 2n+1]$ (note that $\mathbb{R}\cong (2n,2n+1)$), and let $$A=\bigcup_{n\in\mathbb{N}}A_n.$$ If $A$ were $\Sigma^0_n$, then $A_n$ would also be $\Sigma^0_n$ (since the "pieces" of $A$ are "nicely separated"), which isn't the case.
So we need to keep going:
We can keep on going like this, and e.g. define $\Sigma^0_{\omega+17}$, $\Pi^0_{\omega\cdot 2+43}$, $\Sigma^0_{\omega^2+\omega+1}$, and so on. That is, we can define the Borel hierarchy through the countable ordinals. And it turns out that we need all of these:
At the same time it's a good exercise to check that any countable union of sets, each of which is $\Sigma^0_\alpha$ for some countable $\alpha$, is also $\Sigma^0_\alpha$ for some countable $\alpha$. So we get:
That is, the Borel hierarchy stops at $\omega_1$. Note that we can define levels of the Borel hierarchy past the countable ordinals, the point is just that we don't get anything new: the pointclass $\Sigma^0_{\omega_1+17}$ is the same as the pointclass $\Sigma^0_{\omega_1}$. (A "pointclass" is basically a complexity class of sets of reals - "open," "closed," "$F_\sigma$," and so on are examples of pointclasses.)