Let $f(z) = z + z^2$ and let $V = \displaystyle \{z \in \mathbb{C} : |z| < \frac{1}{2}, \frac{3\pi}{4} < arg\{z\} < \frac{5\pi}{4}\}$.
$(a)$ Show that $f(V) \subset V.$
$(b)$ Let $f_n$ be the nth iterate of $f$. Thus $f_1(z) = f(z)$ and $f_{n +1}(z) = f(f_n(z))$ for $n = 1, 2, ...$ For each point $z \in V$, show that $f_n(z) \rightarrow 0$ as $n \rightarrow \infty$.
I tried just doing directly. I really didn't make in progress (I am having trouble with even a). It doesn't seem hard, but I think I am just not seeing it. I got that all points in $f(V)$ belong to the right half circle but that is all. Some help would be great. Thanks.
$V$ is the sector consisting of points with $3\pi/4\le\arg(z)\le 5\pi/4$ and $|z|\le 1/2$, as such it is symmetric with respect to the negative $x$-axis, so it suffices to show the claim for the upper half plane only.
The upper ray of the sector can be parametrized as:
$$z_1(r)=r\exp\left(\frac{3\pi}{4}i\right)\text{, with } 0\le r\le \frac{1}{2}$$
The arc of the sector (down to $-1/2$) can be parametrized as:
$$z_2(\theta)=\frac{1}{2}\exp\left(\theta i\right)\text{, with} \frac{3\pi}{4}\le\theta\le \pi$$
We have now:
$$f(z_1(r))=r\cdot z_1(1/2)-i\cdot r^2$$
and
$$f(z_2(\theta))=z_2(\theta)+\frac{1}{4}\exp(2 i\theta)$$
You can now, either directly show(*) that the above expressions are contractions for the indicated range:
$$|f(z_1(r))|\lt |z_1(r)|\text{, for }0\le r\le \frac{1}{2}$$
and
$$|f(z_2(\theta))|\lt |z_2(\theta)|\text{, for }\frac{3\pi}{4}\le \theta\le \pi$$
or you can argue as follows: $f$ will either be a (possibly rotational) contraction or expansion in the indicated domain. Checking the first iterate of the two end points of the sector we find:
$$f(z_1(1/2))=z_1(1/2)-\frac{i}{4}=-\frac{\sqrt{2}}{4}+\frac{i\sqrt{2}}{4}-\frac{i}{4}\Rightarrow |f(z_1(1/2))|<|z_1(1/2)|$$
and
$$f(z_2(\pi))=\frac{1}{2}\exp(i\pi)+\frac{1}{4}\exp(2 i \pi)=-\frac{1}{4}\Rightarrow |f(z_2(\pi))|<|z_2(\pi)|$$
In particular, $f$ is a contraction in a neighborhood of $0$, so $f(V)\subset V$. But the $\mathit{only}$ fixed point(**) of $f$ is $z=0$, so the sequences defined as:
$$z_n=f^{(n)}(z_1(r_0))\text{, }0\le r_0\le \frac{1}{2}\text{, }n\in\mathbb{N}$$
and
$$z_m=f^{(m)}(z_2(\theta_0))\text{, }\frac{3\pi}{4}\le\theta_0\le \pi\text{, }m\in\mathbb{N}$$
give rise to the decreasing and bounded below (by $0$) sequences $|z_n|$ and $|z_m|$, which will converge to the (only) fixed point of $f$, $z=0$.
(*) left as an exercise.
(**) Note that $z=0$ is a $\mathit{neutral}$ fixed point of the map $f(z)=z^2+z$ because $|f'(0)|=1$, so you cannot use fixed point iteration here.
Here is a visual of what happens on the upper part of the sector. The section below it is conjugate symmetric: