Please consider the following curve integral: $$I:=\int_{\partial B_1(2i)}\frac{e^{z^2}}{2i-z}dz$$ where $$B_r(z_0):=\left\{z\in\mathbb{C}:|z-z_0|<r\right\}$$
Let $\gamma :[a,b]\to\Omega$ denote a piecewise continuously differentiable path in $\Omega$, $\gamma^*:=\gamma([a,b])$ denote its trace, and $f:\gamma^*\to\mathbb{C}$ be continuously $\Rightarrow$ $$\int_\gamma f(z)dz:=\int_a^b\gamma '(t)f(\gamma (t))\;dt$$ is called curve integral of $f$ along $\gamma$.
Using the definition above, we can calculate $I$. However, I'm sure there is an easier way to calculate the integral. I thought about Cauchy's integral theorem or Cauchy's integral formula . I'm new to them and still unsure how they can be applied here.
To use Cauchy's integral formula, in the notation of the wikipedia link you provided, set $f(z)=-e^{z^2}$ and $a=2i$. This gives you $I=-\left.2\pi ie^{z^2}\right|_{z=2i}=-2\pi e^{-4}i$.
You mentioned Cauchy's Integral Theorem, but it requires the integrand to be holomorphic. It isn't, so it doesn't contradict the above.
As an added bonus, one can even infer that $z\mapsto -\dfrac{e^{z^2}}{z-2i}$ doesn't have an antiderivative in $\mathbb C$ (or in any neighborhood of $2i$) for if it did, since the path is closed, the integral would be $0$.