$F(x,y)= \left(\displaystyle\frac{1-y^2}{(1+xy)^2},\displaystyle\frac{1-x^2}{(1+xy)^2}\right)$.
I've been having some troubles to find the potential of $F$.
To find it the idea was finding $\int F dx = g(x,y) +h(y)$, and having $g$ find the function $h$. Unless my calculations have a mistake, this doesn't seem to work.
$$\int Fdx = \int \frac{1-y^2}{(1+xy)^2} dx = (1-y^2)\int \frac{1}{(1+xy)^2}dx\\ = (1-y^2)\left(\frac{-1}{y(1+xy)}+h(y)\right)\\=h(y)-y^2h(y)+\frac{y}{(1+xy)}-\frac{1}{y(1+xy)}.$$
Now I'd have to find the $\partial y$ of the latter equation, and find $h(y)$, but
$$\frac{\partial}{\partial y}\left(h(y)-y^2h(y)+\frac{y}{(1+xy)}-\frac{1}{(1+xy)}\right)\\=h'(y)-2yh(y)-y^2h'(y)+\frac{1}{(1+xy)^2}+\frac{1+2xy}{y^2(1+xy)^2}\\=\frac{y^2+2xy+1}{y^2(1+xy)^2}+h'(y)(1-y^2)+2yh(y).$$
Then I must solve the follwing for $h$
$$\frac{y^2+2xy+1}{y^2(1+xy)^2}+h'(y)(1-y^2)+2yh(y)=\displaystyle\frac{1-x^2}{(1+xy)^2}\\ \frac{x^2y^2+2xy+1}{y^2(1+xy)^2}+h'(y)(1-y^2)+2yh(y)=0.$$
This is where I'm stuck. I don't know how to solve for $h$.
It is much easier if you add the $y$-dependent function after taking a primitive of (the first component of) $F$: $$ \int \frac{1-y^2}{(1+xy)^2}\,dx=\frac{y^2-1}{y(1+xy)}+g(y). $$ Then differentiate with respect to $y$.
(You could also see this as renaming your $h$, like $g=(1-y^2)h$.)
Now, it should be possible to solve this as you do anyways. On your last line, you should cancel $(x^2y^2+2xy+1)$ with $(1+xy)^2$ to find that $$ \frac{1}{y^2}+h'(y)(1-y^2)+2yh(y)=0, $$ or equivalently, with $g$ in place of $h$, $$ \frac{1}{y^2}+g'(y)=0. $$ I'm sure you can take it from here.