This question is taken from Q7 of example sheet 1 of the Complex Methods course at the university of Cambridge. The link for the sheet can be found below:
http://www.damtp.cam.ac.uk/user/examples/B7a.pdf
We define a branch of $f(z)=(z^2-1)^{\frac{1}{2}}$ as follows: Let $[-1,1]$ be the straight line segment connecting $-1$ and $1$ in $\mathbb{C}$. We consider the following branch of $f(z)$: $$ f_{2}:\mathbb{C}-[-1,1] \to \mathbb{C} $$ where $f_{2}(x)=\sqrt{x^{2}-1}$ for $x>1$.
I'm trying to compare the limits of $f_{2}$ above and below the branch cut $[-1,1]$. In order to do so, I notice that $z^{2}-1=(z-1)(z+1)$.
Write $$ z-1=r_{+}e^{i\theta_{+}} $$ and $$ z+1=r_{-}e^{i\theta_{-}} $$ where $\theta_{+} \in (-\pi,\pi)$ and $\theta_{-}\in(0,2\pi)$ based on the branch cut.
My understanding is that just above the brach cut $\theta_{-}=0$ and $\theta_{+}=\pi$ so $(z^2-1)^{\frac{1}{2}}=\sqrt{r_{+}r_{-}}e^{i\frac{\pi}{2}}=i\sqrt{1-x^{2}}$.
Below the branch cut $\theta_{-}=2 \pi$ and $\theta_{+}=-\pi$ so $(z^2-1)^{\frac{1}{2}}=\sqrt{r_{+}r_{-}}e^{i\frac{\pi}{2}}=i\sqrt{1-x^{2}}$.
The answer is wrong though. I'm not sure why and would appreciate any help given.
Let us denote the principal branch of the square root with $S$, i.e. $$ S(re^{i \phi}) = \sqrt r e^{i \phi/2} $$ for $r > 0$ and $-\pi < \phi< \pi$. Then $g(z) = S(z-1)S(z+1)$ is a holomorphic square root of $z^2-1$ on $D = \Bbb C \setminus (-\infty, 1]$ and $g(x) = f_2(x)$ for $x > 1$.
It follows that $f_2(z) = S(z-1)S(z+1)$ for $z \in D$. For $z \to x \in (-1,1)$ converges $z-1 \to x-1 < 0$, so that $$ S(z-1) \to \begin{cases} i \sqrt{1-x} & \text{ from above,} \\ -i \sqrt{1-x} & \text{ from below,} \end{cases} $$ and $z+1 \to x+1 > 0$ so that $S(z+1) \to \sqrt{x+1}$. Then $$ f_2(z) \to \begin{cases} i \sqrt{1-x}\sqrt{x+1} = i\sqrt{1-x^2} & \text{ from above,} \\ -i \sqrt{1-x}\sqrt{x+1} = -i\sqrt{1-x^2}& \text{ from below.} \end{cases} $$
The error in your calculation is that both $\theta_+ = \arg(z-1)$ and $\theta_- = \arg(z+1)$ should be in the range $(-\pi, \pi)$.