Sigma Algebra on Coin Toss

Question

I am trying to study probability space, and so far I have come to point that probability space is defined as $( \Omega, \mathcal{F}, P )$ where $\mathcal{F}$ is the $\sigma-algebra$. I know what $\sigma-algebra$ is, but I am confused that the $\sigma-algebra$ can be easily obtained by the power set of $\Omega$ .i.e. $2^\Omega$. Since a set can have many $\sigma-algebras$, are we only interested in the smallest $\sigma-algebra$ generated by events of interest?

For example, consider tossing a coin twice, and we are interested in finding the probability such that there is at least one head. So, surely $\Omega = (HH, HT, TH, TT)$. But what about $\mathcal{F}?$ One straight forward option is power set, but I am interested in "smallest" part of $\sigma-algebra$ which seems to come very handy in probability theory? Suppose I take $\mathcal{F} = (\emptyset, \Omega, \{HH, HT\}, \{TH, TT\})$. Now with this $\mathcal{F}$, I cannot measure "at least one head" because $(HH,HT, TH)$ is not in $\mathcal{F}$. It definitely means that my $\mathcal{F}$ has to be chosen smartly. How can I help myself here?

Answer 1

A $\sigma$-algebra is a collection of sets that needs to satisfy certain conditions. Yours violates one of the most important ones, namely, the fact that it should be closed under countable unions. (If you have a finite $\Omega$ and you put all the single elements into your algebra, then it necessarily has to be the power set by this axiom.)

Answer 2

A sigma algebra must be closed for countable unions and relative complements.

One such set with that property is the power set of $\Omega$, ie: the set of all subsets for $\Omega$, and when $\Omega=\{{\sf TT, HT, TH, HH}\}$ then that is: $$\begin{align}\mathcal F & = 2^\Omega\\ & =\{\{\}, \{{\sf TT}\},\{{\sf HT}\},\{{\sf TH}\},\{{\sf HH}\},\{{\sf TT, HT}\},\ldots\{{\sf TH, HH}\},\ldots,\{{\sf TT, HT, TH, HH}\}\}\end{align}$$

I am not going to list all $16$ members of the powerset.   However, one of them will be $\{{\sf HT, TH, HH}\}$ .

$~\\~\\~$

---

There is a smaller sigma-algebra over $\Omega$ which contains $\{{\sf HT, TH, HH}\}$, and that is: $$\{\{\},\{{\sf TT}\},\{{\sf HT, TH, HH}\},\{{\sf TT, HT, TH, HH}\}\}$$

Answer 3

If you choose to take a $\sigma$-algebra that is not the power set $\sigma$-algebra, you will necessarily lose information. To see this, let $\Omega = \{x_1, ..., x_n\}$ be a finite set. If $\mathcal{F}$ is a $\sigma$-algebra on $\Omega$ where $\{x_i\} \in \mathcal{F}$ for all $\{x_i\}$, the requirement that $\sigma$-algebras be closed under countable union forces $\mathcal{F} = \mathscr{P}(\Omega)$. Hence, if you choose any $\sigma$-algebra that is not $\mathscr{P}(\Omega)$, you can no longer determine the likelihood of individual outcomes.

However, if you already know the sets in $\Omega$ that you want to be able to measure, then there is a well-defined, smallest $\sigma$-algebra that contains these sets. If $\{E_i\}$ is a collection of subsets of $\Omega$, then the power set $\sigma$-algebra on $\Omega$ is a $\sigma$-algebra containing all $\{E_i\}$. Since the intersection of any number of $\sigma$-algebras on $\Omega$ containing all the $\{E_i\}$ is a $\sigma$-algebra containing all the $\{E_i\}$ the intersection of all $\sigma$-algebras containing the $\{E_i\}$ is a $\sigma$-algebra containing all the $\{E_i\}$ and is clearly the smallest such $\sigma$-algebra. We call this the $\sigma$-algebra generated by the $\{E_i\}$. Qualitatively, this is the $\sigma$-algebra formed by taking complements, finite intersections, and countable unions of members of $\{E_i\}$ until it satisfies all requirements of a $\sigma$-algebra.

In your example, you want to be able to measure the probability of having at least one head. The only set you want to measure in this is now $\{HT, TH, HH\}$. When we go through this process, we obtain the $\sigma$-algebra $$\mathcal{F} = \{\emptyset, \{TT\}, \{HT, TH, HH\}, \{HH, HT, TH, TT\}$$ as the smallest $\sigma$-algebra that makes $\{HT, TH, HH\}$ measurable.