It is well-known that $$\mathbb{P}(A)=\mathbb{E}[\mathbb{1}_{A}]$$
for an event $A \in \Omega$.
However, if we have a $\sigma$-algebra $\mathcal{F}$ then it certainly it is not true that $$\mathbb{P}(A|\mathcal{F})=\mathbb{E}[\mathbb{1}_{A}|\mathcal{F}]$$
since the LHS is a real number, while the RHS is a function.
My question is: Do we have a similar relationship between conditional probability (on a $\sigma$-algebra) and the expectation of an indicator function?
LHS is NOT a number; you can check Durrett Probability Theory and Examples that the second equality is actually correct. (version 4.1, page 191 at the end).
The way that people sometimes use $\mathbb{P}(A \mid B)$ is simply the value of this random variable on the event B, i.e. then it becomes a number.