I am studying complex analysis and I do not yet fully understand branch points and branch cuts. I am trying to figure out how it works by looking at the following:
$z \rightarrow \frac{1}{2i} \log(\frac{1+iz}{1-iz})$
(arctan(z)= $\log(\frac{1+iz}{1-iz})$ )
Now how do I find the branch point and branch cuts?
On
(This answers questions that arose in comments, but is itself too lengthy for a comment.)
If $f:X \to Y$ is a mapping, a "branch of $f^{-1}$" is a mapping $g:Y \to X$ such that $(f \circ g)(y) = y$ for all $y$ in $Y$, a.k.a. a right inverse of $f$. (Note carefully that $(g \circ f)(x) = x$ does not generally hold for all $x$ in $X$, but only for $x$ in the image of $g$. Think of the positive branch of the real square root, or the inverse trig functions.)
In particular, if $U$ is a (usually open) set of complex numbers, a branch of $\log$ in $U$ is a mapping $L:U \to \mathbf{C}$ such that $\exp\bigl(L(w)\bigr) = w$ for all $w$ in $U$. If $L$ is continuous, we call $L$ a continuous branch of $\log$, and say there exists a continuous branch of $\log$ in $U$.
To address the questions of (i) how to see there is no continuous branch of $\log$ on any punctured neighborhood of $0$ or $\infty$ and (ii) there is such a branch near an arbitrary non-zero number, here's a depiction of the complex logarithm:
Writing $z = x + iy = re^{i\theta}$ (with $r>0$, and $x$, $y$, and $\theta$ real) and $w = \log z = (\log r) + i\theta$, the surface has equation $Z = \operatorname{Re}(w) + \operatorname{Im}(w) = (\log r) + \theta$ (with $Z$ denoting the height coordinate in $\mathbf{R}^3$).
Geometrically, this surface is a "helicoid" swept out by the graph $\operatorname{Re}(w) = \log x$ by simultaneously revolving at unit angular speed about the $Z$ axis and translating at unit speed along the $Z$ axis. It consists of infinitely many "sheets" (three of which are shown), joined one to the next. A branch cut is the image $\gamma$ of a piecewise $C^1$ curve in $\mathbf{C}$ joining $0$ to $\infty$. If you remove points on the surface lying above $\gamma$, the surface separates into infinitely many graphs, each the geometric representation of a continuous branch of $\log$ on $U = \mathbf{C} \setminus \gamma$.
The green curve is a "lifting" of two full turns of the unit circle in the $z$ plane; the fact that the lift is not a closed curve manifests the fact that there is no continuous branch of $\log$ in a punctured neighborhood of $0$. The picture near $\infty$ is congruent, since "$\log(1/z) = -\log z$", with the understanding that each side is multi-valued.
Hint: for every $a,b\in\mathbb{C}$, find those $z\in\mathbb{C}$ such that $\frac{z-a}{z-b}\in\mathbb{R}$. Then find exactly when the given fraction is positive, and when it's negative.
Edit: for any $u,w\in\mathbb{C}$ we have $\arg{\frac{u}{w}} = \arg{u}-\arg{w}$. It follows that the quotient is real iff $u,w$ have the same argument (they 'point' in the same direction) or opposite arguments. How can we apply that to the first hint?
Edit: the next part is more explicit, and I'll try to give some intuition based on your familiarity with Möbius transformation (however, if you're not done thinking about the problem, you might wish to delay reading it). As mentioned by @AndrewD.Hwang in the comments above, an analytic branch of logarithm exists in any simply connected domain not including zero. Put differently, when you think of the complex plane as the Riemann sphere (infinity as the 'north' pole), the logarithm has branch points at the poles (zero and infinity), and removing any arc connecting both poles (that arc becomes the branch cut) will yield a simply connected surface on which an analytic branch of logarithm indeed exists.
Now, $\varphi(z) = \frac{1+iz}{1-iz}$ is a Möbius transformation. This means that it's conformal on the entire Riemann sphere (if you'd like, it 'distorts' the sphere in general, but locally it behaves very similarly to shifts and rotations). Note that it maps $i\mapsto 0$, $-i\mapsto \infty$, implying that the image of any arc connecting $\pm i$ under $\varphi$ is an arc connecting $0,\infty$, and indeed $\pm i$ are the branch points of $\arctan := \log\circ\varphi$, and any arc connecting them would be a branch cut (in other words, removing such an arc would yield a domain in which an analytic branch of $\arctan$ exists).
Now, as AndrewD.Hwang also mentioned, the standard 'choice' of branch cut for logarithm is the non-positive real line. It is here that my original hint could help us, as it allows us to find the arc mapped by $\varphi$ to that cut. As I said, for any $a,b\in\mathbb{C}$ we have $\frac{z-a}{z-b}\in\mathbb{R}$ iff $(z-a),(z-b)$ point in similar or opposite directions. This happens exactly when $z$ lies on the unique line passing through $a,b$, and the quotient is negative iff $z$ lies on the segment connecting $a,b$ (because that is when the arrows from $a,b$ to $z$ point in opposite directions). More rigorously, one notes: $$\frac{z-a}{z-b}=t \iff z-a=t(z-b) \iff (1-t)z = a - tb = (1-t)a - t(b-a)\\ \iff z = a + \frac{t}{t-1}(b-a),$$ and indeed $\frac{t}{t-1}\in(0,1)$ ($z$ lies between $a,b$) iff $t$ is non-positive.
Finally, note that $$\varphi(z) = \frac{i(z-i)}{i(-i-z)} = -\frac{z-i}{z+i}$$ is a non-positive real exactly when $z$ lies on the imaginary axis but not between $\pm i$, i.e. on $$\{it\mid t\in\mathbb{R}, |t|\geq 1\}.$$ (This, you see, is an arc in the Riemann sphere connecting $\pm i$, which has the 'nice' property of passing through infinity--the point on the sphere that we 'dislike' to work with. Or, if you'd like, it's the only arc connecting $\pm i$ which doesn't pass through the real line, allowing us to truly extend the familiar $\arctan$ on reals.)