I'm working on a problem set, so I'm not looking for a solution, but just maybe a pointer on where I'm going wrong.
I want to use the holomorphic functional calculus to determine the square root of the matrix $$T=\begin{bmatrix}1&1\\0&1\end{bmatrix}.$$ The spectrum of this matrix is just $\{1\}$, so a convenient curve enclosing the spectrum is $\gamma(t)=2e^{it}$ for $t\in\left[-\pi,\pi\right)$. Now I know that for $f(z)$ holomorphic (at least on a region containing both the spectrum and $\gamma$), the operator
$\hspace{20pt}f(T)=\frac{1}{2\pi i}\int_\gamma f(\xi)(\xi I-T)^{-1}d\xi=\frac{1}{2\pi i}\int_{-\pi}^\pi f(2e^{it})(\xi I-T)^{-1}\cdot 2ie^{it}dt$
is well-defined. When I then plug in a polynomial for $f$, I get the right answer. However, when I plug in the square root, which I'm taking to be $\sqrt{\gamma(t)}=\sqrt{2}e^{it/2}$ so that the branch cut is along the negative real axis, I get a nonsensical answer:
$\hspace{20pt}\left(\frac{20+\sqrt{10}\pi-2\sqrt{10}\arctan\sqrt{10}}{5\sqrt{2}\pi},\frac{10\sqrt{2}+11\sqrt{5}\pi-22\sqrt{5}\arctan\sqrt{10}}{110\pi};0,\frac{20+\sqrt{10}\pi-2\sqrt{10}\arctan\sqrt{10}}{5\sqrt{2}\pi}\right).$
Of course I expect $\sqrt{T}=\begin{bmatrix}1&1/2\\0&1\end{bmatrix}$. The only thing that seems different is that the square root is a multi-valued function whereas polynomials are single-valued, so I'm assuming my error is failing to take something along these lines into account. (I should also not that I'm doing the integral in Mathematica, so maybe there's some problem with doing that.)
The problem is that your circle is unnecessarily big, and you are hitting $0$ where the square root is not analytic.
If you use the circle $1+e^{it}/2$ and the analytic expression for the square root in the disk of radius 1 around 1 $$ f(z)=\sum_{k=0}^\infty {1/2 \choose k}\,(z-1)^k, $$ you will get the right values. The uniform convergence will allow you to integrate term by term, so the computations are very simple.