Meaning of measurableness

Question

I've done courses on measure theory, advanced stochastic processes, etc.--years and years of using the notion of 'measurableness'. I've now come to the conclusion that, in fact, I do not understand this notion.

Yes, I 'understand' the "mathematical" meaning (i.e., the definition), it's very simple: The inverse images of (Borel-)measurable sets need to be in the sigma algebra. In fact mathematically there isn't anything to understand, it's just a definition. Yet there's a difference between 'understanding' a definition mathematically, and really understanding the semantics on the deepest level (which I'd call the 'true' meaning).

Concretely, let's focus on probability spaces. Let $(\Omega, \mathcal{F}, P)$ be a probability space. Let $\mathcal G \subseteq \mathcal F$ be a sub-sigma-algebra.

(1) What is the meaning of "the information contained in $\mathcal G$"?

Let $X \colon \Omega \to \mathbb{R}$ be a function. Then $X$ is called a random variable iff $X$ is $\mathcal F$-measurable. This means that, for any Borel set $B$, we have that $(X \in B)$ lies in $\mathcal F$. The reason behind this definition is that, for any such Borel set $B$, we want to be able to compute the probability that $X \in B$ (therefore we must demand all such to be part of the sigma-algebra). Let's suppose that $\sigma(X) \subseteq \mathcal G$, so that $X$ is even measurable w.r.t. the smaller sigma algebra.

(2) How can the "information" in $\mathcal G$ determine the value of $X$?

$\mathcal G$ is some smaller sigma algebra. In which sense does the information in $\mathcal G$ determine the value of $X$? I don't see that at all. In fact, in which sense does the information in $\mathcal F$ determine the value of $X$!?

Despite years of thinking and searching for answers online (including numerous MSE posts), the following question is pertinent:

(3) What is the link between sigma algebras and information?

(NB: I know e.g. What does it mean by saying 'a random variable $\mathit X$ is $\mathcal G$-measurable'? and Problem with intuition regarding sigma algebras and information and many others are similar/a duplicate of this, but it didn't help me solve my problems--all questions there were left unanswered, completely.)

TL;DR: I do not understand the link between sigma algebras and "information". I would appreciate any help. Right now I'm totally stuck. If I can't clear this up I basically have to drop out of the course and find a different career path.

Answer 1

Here is an intuitive answer which may address your concern about $\sigma$-algebras and information.

Assume that only $3$ mutually exclusive events may happen at time $T$. Let these be denoted by $\omega_1$, $\omega_2$, $\omega_3$. The probabilities of these events are estimated to be $p_1, p_2, p_3 \in [0, 1]$ such that $p_1+p_2+p_3=1$. At time $T$, based on the occurrence of a particular event, John, Jack and Jane will take actions $X$, $Y$, $Z$ from a set of possible actions $A=\{0, 1, 2\}$.

Now consider the task of modelling $X$, $Y$, $Z$ mathematically, such that you can speak of the probability of a particular action taken. Finding the right model for $X$, $Y$, $Z$ will depend on the possible restrictions that John, Jack and Jane face at time $T$. Assume that at time $T$, the individual circumstances of John, Jack and Jane are as follows:

At time $T$, John will know exactly which of the mutually exclusive events $\omega_1, \omega_2, \omega_3$ has occurred and will take the action $1$, $2$, $3$, respectively.

At time $T$, Jack will only be able to tell whether $\omega_1$ has occurred or not. So if $\omega_1$ has occurred, he will take action $1$. If $\omega_1$ has not occurred, Jack will know that either $\omega_2$ or $\omega_3$ has occurred, but he will not know which one exactly, and, in either case, he will take action $2$.

At time $T$, Jane will only be able to tell whether $\omega_2$ has occurred or not. So if $\omega_2$ has occurred, she will take action $1$. If $\omega_2$ has not occurred, Jane will know that either $\omega_1$ or $\omega_3$ has occurred, but she will not know which one exactly, and, in either case, she will take action $3$.

Now, let us come up with a suitable mathematical model for the taken actions $X$, $Y$, $Z$. This will be accomplished by designing an individual probability space $(\Omega, \mathcal{F}, \mathbb{P})$ for the random variables $X$, $Y$, $Z$. In all the three cases, $\Omega$ will be given by $\{\omega_1, \omega_2, \omega_2\}$. However, the $\sigma$-algebra $\mathcal{F}$ should model the information accessible at time $T$ to the person under question. By saying information, we mean the events that are observable at time $T$ to the person under question.

John. According to the description, action $X$ taken by John is defined as follows: $X(\omega_1)=1$, $X(\omega_2)=2$, $X(\omega_3)=3$. Since John has complete information at time $T$, i.e., he is able to distinguish which of the mutually exclusive events $\omega_1$, $\omega_2$ and $\omega_3$ has occurred, the corresponding sigma-algebra $\mathcal{F}$ should reflect this fact. Therefore $\mathcal{F}$ should contain all the individual events $\{\omega_1\}$, $\{\omega_2\}$, $\{\omega_3\}$. Of course, John is also able to observe the event "either $\omega_1$ or $\omega_2$ has occurred", which is modelled by including the union of $\{\omega_1\}$ and $\{\omega_2\}$, given by $\{\omega_1, \omega_2\}$, into $\mathcal{F}$. Through a similar line of thought, we see that $\mathcal{F}$ has to be the power set of $\Omega$: $$ \mathcal{F}_1 = \mathscr{P}(\Omega) = \sigma(X). $$ The mutually exclusive events $\{\omega_1\}$, $\{\omega_2\}$, $\{\omega_3\}$ generate $\mathcal{F}$, and the values of $X$ are captured/determined by the values on these generating events.

Jack. According to the description, action $Y$ taken by Jack is defined as follows: $Y(\omega_1)=1$, $Y(\omega_2)=2$, $Y(\omega_3)=2$. Since at time $T$, John is able to distinguish the events "$\omega_1$ has occurred" and "either $\omega_2$ or $\omega_3$ has occurred", we include $\{\omega_1\}$ and $\{\omega_2, \omega_3\}$ in $\mathcal{F}$. Of course John is also able to tell whether either of the two aforementioned events has occurred, which is reflected by including $\{\omega_1, \omega_2, \omega_3\}$ in $\mathcal{F}$. Hence, $$ \mathcal{F}_2 = \{ \{\omega_1, \omega_2, \omega_3\}, \{\omega_2, \omega_3\}, \{\omega_1\}, \emptyset \} = \sigma(Y). $$ The mutually exclusive events $\{\omega_1\}$, $\{\omega_2, \omega_3\}$ generate $\mathcal{F}$, and the values of $Y$ are captured/determined by the values on these generating events.

Jane. According to the description, action $Z$ taken by Jane is defined as follows: $Z(\omega_1)=3$, $Z(\omega_2)=1$, $Z(\omega_3)=3$. Since at time $T$, Jane is able to distinguish the events "$\omega_2$ has occurred" and "either $\omega_1$ or $\omega_3$ has occurred", we include $\{\omega_2\}$ and $\{\omega_1, \omega_3\}$ in $\mathcal{F}$. Of course Jane is also able to tell whether either of the two aforementioned events has occurred, which is reflected by including $\{\omega_1, \omega_2, \omega_3\}$ in $\mathcal{F}$. Hence, $$ \mathcal{F}_3 = \{ \{\omega_1, \omega_2, \omega_3\}, \{\omega_1, \omega_3\}, \{\omega_2\}, \emptyset \} = \sigma(Z). $$ The mutually exclusive events $\{\omega_2\}$, $\{\omega_1, \omega_3\}$ generate $\mathcal{F}$, and the values of $Y$ are captured/determined by the values on these generating events.

It is easy to see that $X, Y, Z$ are all $\mathcal{F}_1$-measurable, since $\mathcal{F}_1$ is the $\sigma$-algebra with complete information. But $X$ is not $\mathcal{F}_2$-measurable, i.e., not all the events associated with $X$ are observable in $\mathcal{F}_2$. For example, $\mathcal{F}_2$ does not contain $\{\omega_2\}$, and one cannot speak of the probability of $X=2$ on the probability space $(\Omega, \mathcal{F}_2, \mathbb{P})$.

Answer 2

A sub-$\sigma$-algebra $\mathcal{G}$ represents "partial information" in the following sense. For each $G\in\mathcal{G}$, an observer (who doesn't know which $\omega$ has been drawn) knows if $\omega\in G$ or $\omega\in G^{c}$. This is also related to conditional expectation, i.e., a finer $\mathcal{G}$ gives a better prediction of some unknown quantity $Y$ living on $(\Omega,\mathcal{F},\mathsf{P})$. Assuming that $Y\in L^2$, and $\mathcal{G}'\subset\mathcal{G}''$, \begin{align} \mathsf{E}[(Y-\mathsf{E}[Y\mid \mathcal{G}'])^2\mid \mathcal{G}'']&=\mathsf{E}[(Y-\mathsf{E}[Y\mid \mathcal{G}'']+\mathsf{E}[Y\mid \mathcal{G}'']-\mathsf{E}[Y\mid \mathcal{G}'])^2\mid \mathcal{G}''] \\ &=\mathsf{E}[(Y-\mathsf{E}[Y\mid \mathcal{G}''])^2\mid \mathcal{G}''] +(\mathsf{E}[Y\mid \mathcal{G}'']-\mathsf{E}[Y\mid \mathcal{G}'])^2. \end{align} That is, $\mathsf{E}[(Y-\mathsf{E}[Y\mid \mathcal{G}'])^2]\ge \mathsf{E}[(Y-\mathsf{E}[Y\mid \mathcal{G}''])^2]$.

On the other hand, the interpretation of $\sigma$-algebras as information is purely informal as pointed out by Billingsley (Example 4.9 in his Probability & Measure, 1986). First, he defines partitions of $\mathcal{G}$, i.e., $\omega$ and $\omega'$ are $\mathcal{G}$-equivalent if $1_G(\omega)=1_G(\omega')$ for every $G\in \mathcal{G}$. Sets of $\mathcal{G}$-equivalent points is the partition of $\mathcal{G}$. Our observer knows the equivalence class to which $\omega$ belongs (which we interpret as information).

Now, let $([0,1],\mathcal{B}_{[0,1]},\mathsf{P})$ be the probability space, where $\mathsf{P}$ is the Lebesgue measure, and let $\mathcal{G}=\sigma\{\{\omega\}:\omega\in [0,1]\}$, i.e., $\mathcal{G}$ is the countable/co-countable $\sigma$-algebra. Notice that $\mathcal{G}$ is independent of $\mathcal{B}_{[0,1]}$ because all the events in $\mathcal{G}$ are trivial. Thus, it does not contain any information about the sets in $\mathcal{B}_{[0,1]}$ in the sense that $\mathsf{P}(B\mid \mathcal{G})=\mathsf{P}(B)$. On the other hand, the $\mathcal{G}$-partition consists of the singletons, i.e., it contains "all the information".