I am experiencing some serious confusion with instantons..
In Nakahara's "Geometry, Topology and Physics" p.65, or Weinberg "Quantum Theory of Fields Vol 2 p.450~452, they say that on $\mathbb{R}^4$, with $r:=\lvert x \rvert$ for $x \in \mathbb{R}^4$, the instanton solution is given by
\begin{equation} \mathcal{A}_\mu=\frac{r^2}{\lambda^2+r^2}U(x)^{-1} \partial_\mu U(x) \end{equation}
for $U(x):=\frac{1}{r}(x_4-ix_j \sigma_j)$, where $\sigma_j$'s are Pauli matrices and $\lambda$ is an arbitrary positive number that represents the size of the instanton.
Then, the Nakahara book says the corresponding field strength tensor is \begin{equation} \mathcal{F}_{\mu \nu}:=\frac{4\lambda^4}{\lambda^2+r^2} \sigma_{\mu \nu} \end{equation} with $\sigma_{i j}:=\frac{1}{4i}[\sigma_i, \sigma_j]$ and $\sigma_{i0}=-\sigma_{0i}=\frac{1}{2}\sigma_i$.
However, the serious problem is that the integral \begin{equation} \int d^4x \text{ tr} [\mathcal{F}_{\mu \nu}\mathcal{F}^{\mu \nu}] \end{equation} on $\mathbb{R}^4$ does NOT converge with the above $\mathcal{F}_{\mu \nu}$..... In later chapters, the Nakahara book suddenly changes the domain of integral to $S^4$ and proceeds the calculation and gets the well-known result of the Index Theorem.
But, in p.65 of "Geometry, Topology and Physics" or in p.450~452 of Weinberg's "Quantum Theory of Fields", it seems so obvious that they are denoting the domain of integration as $\mathbb{R}^4$; for example, eqn (23.5.2) of Weinberg has $r \to \infty$ and eqn (23.5.10) of Weinberg uses the notation $(d^4x)_E$, which means the integration on Euclidean $\mathbb{R}^4$....Nakahara (1.301a) also uses the notation $r \to \infty$, which clearly implies $\mathbb{R}^4$.
What am I missing? It is extremely confusing and frustrating...Could anyone please clarify??
Your formula for $F_{\mu\nu}$ is not correct: $$ F_{\mu\nu} = \frac{4\lambda^2\sigma_{\mu\nu}}{(r^2 +\lambda^2)^2} $$ and the integral is $$ \int tr {F^2} \propto \int \frac{r^3 dr}{(r^2+\lambda^2)^4} $$ which is convergent.