Consider the following function $$F(x)=x,~ for~0<x<1 ~; 0~otherwise.$$
I was trying to compute the inverse fourier transformation as (by using wolframalpha) $$f(y)=\int_{R} F(x)e^{2 \pi i x y}dx=\int_0^1 xe^{2 \pi i x y}dx= \frac{-1+e^{2 \pi i x}(1-2 \pi i x)}{4 \pi ^2 x^2}.$$ Now, can I conclude that $$f(x)=\frac{-1+e^{2 \pi i x}(1-2 \pi i x)}{4 \pi ^2 x^2}$$ (here, I am not sure, if I should define $f$ by $0$ outside $(0,1)$ or not) and that FT of $f$ is $F$ (the FT of the result expresion differs from the origin)?
Yes. Just perform the integration by parts
$$\begin{align*} f(y) &= \int_{-\infty}^\infty F(x)e^{2\pi i yx} dx \\ \\ &= \int_0^1 xe^{2\pi i yx} dx \\ \\ &= \dfrac{xe^{2\pi i yx}}{2\pi i y}\biggr{\rvert}_0^1 - \int_0^1 \dfrac{e^{2\pi i yx}}{2\pi i y} dx\\ \\ &= \dfrac{e^{2\pi i y}}{2\pi i y} - \dfrac{e^{2\pi i yx}}{\left(2\pi i y\right)^2}\biggr{\rvert}_0^1 \\ \\ &= -\dfrac{2\pi i y e^{2\pi i y}}{4\pi^2 y^2} + \dfrac{e^{2\pi i y}}{4\pi^2y^2} - \dfrac{1}{4\pi^2y^2} \\ \\ &= \dfrac{-1 + e^{2\pi i y}\left(1-2\pi i y \right)}{4\pi^2y^2}\\ \end{align*}$$
No, the resulting function is not limited in domain; $y \in (-\infty, \infty)$
It is worth noting that $f(y)$ has a singularity at $y=0$. You'll have to use L'Hopital's rule to check if the singularity is removable or not.