In this video the man says that `a measure $\mu$ on $\Omega$ with $\sigma$-algebra $\mathscr{A}$ is a function $\mu \colon\mathscr{A} \rightarrow[0,\infty]$ s.t. [...].
What does that mean? The part that is confusing me is that infinity is not a real!
On
The set $[0,+\infty]$ should be understood as $$\Bbb R_{\geq0}\cup\{+\infty\}$$
Recall how we also write the extended reals $$\Bbb R^*=\Bbb R\cup\{-\infty,+\infty\}$$
No one would be worried if one wrote, say
Define $f:[0,\infty]\to\Bbb R^*$ as $$f(x)=\begin{cases}-\infty&x=0\cr \log x&x\in\Bbb R_{>0}\cr +\infty &x=+\infty\end{cases}$$
You're right when you say that $\infty \notin \textbf{R}$, but it is often convenient, especially when specifying the domain of a function, to consider the $\textit{extended reals}$ $\textbf{R} \cup \{ \infty \}$. The point of a measure $\mu$ is to specify, in a precise way, the size of something, and we have to allow for that size to be infinite.
Consider an analagous example: Let $ \mu : \cal{P}(\textbf{N}) \to \textbf{N} \cup \{ \infty \}$ be the function whose domain is all subsets of the natural numbers and whose range is the natural numbers plus infinity (we might call them the extended natural numbers). This function gives the size of any given subset. Then
$\mu(\emptyset)=0$
and
$\mu ( \{1,3,4,11\}) = 4$
while
$\mu (\{ n \in \textbf{N} : n \equiv 0\mod 2 \})=\infty$.