Is it possible to find a probability space $(\mathbb{C}, \Sigma, \mathbb{P})$, where $\mathbb{C}$ is the complex plane and $\Sigma$ is the Borel sigma algebra, and a measurable function $F$ such that,
$$ \int_\mathbb{C} d \mathbb{P}(t) F^n(t) = \begin{cases} 0 & \mbox{if $n \in \mathbb{N} \, \, : \, \, n \not\in \{0,2\}$} \\ \neq 0 ~& \mbox{if $n=2$}. \end{cases} $$
Yes, this is possible. One example goes as follows.
Take $F(t) \equiv t$, and let $\mathbb{P}$ be the probability measure on the unit circle in $\mathbb{C}$ given by $$d \mathbb{P} (\theta) = \frac{1 + \cos 2 \theta}{2 \pi} \, d\theta.$$
Note that using this measure, the integrals in question can be written as $$\int d \mathbb{P} (t) \, F^n (t) = \int d \mathbb{P} (t) \, t^n = \frac{1}{2 \pi} \int_{0}^{2 \pi} e^{i n \theta} (1 + \cos 2 \theta) \, d\theta.$$
Using the orthogonality properties of $\{ \sin n \theta \}$ and $\{ \cos n \theta \}$ on $[0, 2\pi]$ then establishes the desired conditions.