By wikipedia, suppose $A, B$ are left $R$-modules, one way to calculate $Ext_{R}^{1}(A, B)$ is to regard it as equivalent class of module extension of $A$ by $B$, in the sense that the diagram
$$\require{AMScd}\begin{CD} 0 @>>> B @>>> E @>>> A @>>> 0 \\ @. @| @VfVV @| \\ 0 @>>> B @>>> E' @>>> A @>>> 0 \end{CD}$$
is commutative. Further, the identity element in $Ext_{R}^{1}(A, B)$ is regarded as split extension $E=A\oplus B.$
Now, let's consider $Ext_{\mathbb Z}^{1}(S^1, \mathbb Z)$. To be specific, let $S^1=\{e^{2\pi i\theta}|0\le \theta\lt1\}$, which forms a group under complex multiplication $e^{2\pi ia}\cdot e^{2\pi ib}=e^{2\pi i(a+b)}$. Now, consider the projection map $$\phi: \mathbb R\to S^1$$ $$x\to e^{2\pi i[x]}.$$
with $\ker \phi=\mathbb Z$. And this gives a short exact sequence for abelian groups, and also for $\mathbb Z$-modules:
$$0\to \mathbb Z\to \mathbb R\xrightarrow{\phi}S^1\to 0.$$
Since taken any rational number $a$, which has finite order in $S^1$, so if $\sigma: S^1\to \mathbb R$, then $\sigma(a)=0$, this implies that there is no homomorphism $\sigma: S^1\to \mathbb R$, such that $\phi\circ \sigma=id_{S^1}.$Therefore the above extension is not split. Therefore we have shown that $Ext_{\mathbb Z}^{1}(S^1, \mathbb Z)$ is non-trivial. But, how can I compute the whole $Ext_{\mathbb Z}^{1}(S^1, \mathbb Z)$?
$\text{Ext}^1(-, -)$ sends direct sums in the first variable to direct products, and as mentioned in the comments, as an abstract abelian group
$$S^1 \cong \left( \bigoplus_X \mathbb{Q} \right) \oplus \mathbb{Q}/\mathbb{Z}$$
where $X$ is uncountable. So it suffices to compute $\text{Ext}^1(\mathbb{Q}, \mathbb{Z})$ and $\text{Ext}^1(\mathbb{Q}/\mathbb{Z}, \mathbb{Z})$. The former group turns out to be an uncountable-dimensional rational vector space, while the latter is a bit more complicated: see these notes and this paper for details.