Let $\Omega=[0,1)$ with $\mathcal{B}([0,1))$ and $\lambda$ measure. Calculate $\mathbb{P}(A|\mathcal{G}) = \mathbb{E}(\mathbb{1}_A|\mathcal{G}$) if $A=(\frac{1}{8},\frac{3}{4})$ and $\mathcal{G} = \sigma([0, \frac{1}{4}),[\frac{1}{4},1))$.
$$ \mathbb{E}(\mathbb{1}_A|[0, \frac{1}{4}))=\frac{1}{\frac{1}{4}}\int_{[0,\frac{1}{4})}\mathbb{1}_Ad\lambda = 4 \int_{\frac{1}{8}}^{\frac{1}{4}}1dx = \frac{1}{2} $$
$$ \mathbb{E}(\mathbb{1}_A|[\frac{1}{4}, 1))=\frac{1}{\frac{3}{4}}\int_{[\frac{1}{4},1)}\mathbb{1}_Ad\lambda = \frac{4}{3} \int_{\frac{1}{4}}^{\frac{3}{4}}1dx = \frac{2}{3} $$
So it equals to $\frac{1}{2}$ for $\omega \in [\frac{1}{4}, 1)$ and $\frac{2}{3}$ otherwise.
What bothers me that they don't sum up to 1.
I already know that $\mathbb{E}(X|\mathcal{G})$ is a r.v. and does not necessarily take values only from $[0,1)$, but there was equation $\mathbb{P}(A|\mathcal{G}) = \mathbb{E}(\mathbb{1}_A|\mathcal{G})$ which kind of implies it being probability in this specific case. Is that correct answer or I miscalculated something?
Your calculation is fine.
The values of $\mathbb{P}(A|\mathcal{G})(\omega)$ need to satisfy $\mathbb{E}[\mathbb{P}(A|\mathcal{G})] = \mathbb{P}(A),$ so we shouldn't expect them to sum to $1$.
Rather, for almost every $\omega \in \Omega$ we would have $\mathbb{P}(\cdot|\mathcal{G})(\omega)$ is a probability, so that$$\mathbb{P}(A|\mathcal{G}) + \mathbb{P}(\Omega\setminus A|\mathcal{G}) = 1$$ sums to $1$.