I try to understand how "importance resampling" can be formalized. The idea is the following: Given a target distribution density $p$ and samples $Y_1,\ldots,Y_k$ drawn from a $q$, set weights $w_i:=\frac pq(Y_i)$ and sample an index $I$ proportial to $\frac{W_i}{\sum_{j=1}^kW_j}$ to obtain a sample $X:=Y_I$. Now the claim is that as $k\to\infty$, then $X$ is distributed proporital to $p$ ... But how do we formalize the sampling process above and show that the claim about $k\to\infty$ is actually true? (The claim is made in this work on p. 33)
In order to formalize this, let
- $(E,\mathcal E)$ be a measurable space;
- $\lambda$ be a measure on $(E,\mathcal E)$;
- $p,q:E\to[0,\infty)$ be $\mathcal E$-measurable with $\lambda p=\lambda q=1$;
- $(\Omega,\mathcal A,\operatorname P)$ be a probability space;
- $k\in\mathbb N$;
- $Y_i$ be an $(E,\mathcal E)$-valued random variable on $(\Omega,\mathcal A,\operatorname P)$ and $$W_i:=\frac pq(Y_i);$$
- $\zeta$ denote the counting measure on $\{1,\ldots,k\}$ and $$C(\omega,i):=\frac{W_i(\omega)}{\sum_{j=1}^kW_j(\omega)}$$ for $\omega\in\Omega$ and $i\in\{1,\ldots,k\}$.
But I don't know how the random variable $I$ should be defined ...