I'm reading a Reinforcement Learning book where the following is stated.
$$ Q_t(a)=\frac{\sum_{i=1}^{t-1} R_i \cdot 1A_t=a }{\sum_{i=1}^{t-1} \cdot 1A_t=a} $$
I understand the summing section. So for example, ignoring
$$ 1A_i=a $$
if i: { 1,2,3,4 } and t=3 then qt(a) = sum(1, 2) / sum( 1, 2).
How does
$$ 1A_i=a $$
impact the calculation?
$1_{A_i=a}$ is a notation for the characteristic function of the set $\{i; A_i=a \}$, that is: $$1_{A_i=a}=\begin{cases} 1, & \text{if $i\in $ } \{i; A_i=a \}\\ 0, & \text{if $i \notin \{i; A_i=a \}$} \end{cases}$$
Basically, it means you should sum only over the indexes $i$ such that $A_i=a$.