Mathematica gives $-k \sqrt{\frac{\pi}{2}} \text{sgn}(k)$ as the Fourier transform of $1/x^2$ (i.e., the result of the command FourierTransform[1/x^2, x, k]). And the Fourier transform of $-k \sqrt{\frac{\pi}{2}} \text{sgn}(k)$ is $1/x^2$ (FourierTransform[-k Sqrt[Pi/2] Sign[k], k, x]). I found this by chance while playing with Mathematica, but I cannot understand the result. The definition of Fourier transform $\int_{-\infty}^\infty f(x) e^{-ikx} dx$ does not converge for the functions, and even Mathematica reports that the integrals does not converge. I read the documentation of FourierTransform but could not find any relevant information. In what sense are they Fourier transform of each other?
I found an almost same question (What does the Fourier transform of $1/x^2$ mean?), but the answers looks not so complete. The answers mention "tempered distribution" and so I read some basic materials about it, but $1/x^2$ is not a tempered distribution as some comments on the answers already pointed. Is the notion of the Fourier transform of tempered distributions really relevant to my question?
As you noticed, the function $\frac{1}{x^2}$ cannot be identified as a tempered distribution in the usual way as it is not locally integrable.
Here the symbol "$\frac{1}{x^2}$" has another meaning in the context of tempered distributions, which is formalized by the concept of Hadamard finite-part integral.
That is we define a linear map \begin{align*}\frac{1}{x^2}:\mathcal{S}(\mathbb{R})&\to \mathbb{R}\\ \varphi&\mapsto \lim_{\varepsilon \to 0}\left[\int_{|x|\geq \varepsilon}\frac{\varphi(x)}{x^2}dx-2\frac{\varphi(0)}{\varepsilon}\right] \end{align*} The function is well defined: since $\varphi$ is smooth by the mean value theorem we may write $$\varphi(x)=\varphi(0)+x\varphi'(\xi(x)),\qquad \xi(x)\in [0,x] $$ so that \begin{align*}\int_{|x|\geq \varepsilon}\frac{\varphi(x)}{x^2}dx-2\frac{\varphi(0)}{\varepsilon}&=\varphi(0)\left(\int_{|x|\geq \varepsilon}\frac{1}{x^2}dx-\frac{2}{\varepsilon}\right)+\int_{|x|\geq \varepsilon}\frac{\varphi'(\xi(x))}{x}dx=\\ &=\int_{|x|\geq \varepsilon}\frac{\varphi'(\xi(x))}{x}dx \end{align*} and applying mean value again, with $\varphi'(\xi(x))=\varphi'(0)+\xi(x)\varphi''(\eta(x))$, $$\int_{|x|\geq \varepsilon}\frac{\varphi'(\xi(x))}{x}dx=\varphi'(0)\int_{|x|\geq \varepsilon}\frac{1}{x}dx+\int_{|x|\geq \varepsilon}\varphi''(\eta(x))\frac{\xi(x)}{x}dx=\int_{|x|\geq \varepsilon}\varphi''(\eta(x))\frac{\xi(x)}{x}dx $$ Since $\xi(x)\in [0,x]$ for all $x$, we have $\frac{\xi(x)}{x}\in [0,1]$ and so the last integral converges as $\varepsilon \to 0$, so that $$\frac{1}{x^2}(\varphi)=\int_{-\infty}^{+\infty}\varphi''(\eta(x))\frac{\xi(x)}{x}dx $$
Now to prove that the map $\frac{1}{x^2}$ is a tempered distribution, suppose $\varphi_n\to 0$ in $\mathcal{S}(\mathbb{R})$. Then in particular $\varphi_n''(\eta(x))\to 0$ in $L^1(\mathbb{R})$, and so $$\left|\frac{1}{x^2}(\varphi_n)\right|\leq \int_{-\infty}^{+\infty}|\varphi_n''(\eta(x))|dx\to 0$$
and therefore its Fourier transform is well-defined as the Fourier transform of a tempered distribution.