Matthew Szudzik's mapping function is another approach on mapping $\mathbb{N}\times\mathbb{N}$ to $\mathbb{N}$, however, I have trouble finding the inverse of it. The function is $$f(a, b)=\left\{\begin{array}{ll}a^2+a+b&\mbox{, if }a\geq b\\a+b^2&\mbox{, else}\end{array}\right.$$
He states that the inverse of it is $$f^{-1}(c)=\left\{\begin{array}{ll}(c-\lfloor\sqrt{c}\rfloor^2,\lfloor\sqrt{c}\rfloor)&\mbox{, if } c - \lfloor\sqrt{c}\rfloor^2<\lfloor\sqrt{c}\rfloor\\(\lfloor\sqrt{c}\rfloor, c-\lfloor\sqrt{c}\rfloor^2-\lfloor\sqrt{c}\rfloor)&\mbox{, else}\end{array}\right.$$
But, how did he find it?
$f(a,b) = a^2 + a + b=c; a\ge b$
$f(a,b) = a+b^2=c; a< b$.
So make wild guesses. If $a' = -b'$ we have $f(a',b) = a'^2 = c$ so $g(c) = (\sqrt{c}, -\sqrt{c})$ will work if $c$ is a perfect square. But obviously $c$ needn't be a perfect square. But we can be close.
Suppose $n^2 \le c < (n+1)^2$ or in other words $n \le \sqrt{c}< n+1$. Then $c = n^2 + n + (c-n^2 - n)$. If $c-n^2 -n \le n$ we'd have $f(n,(c-n^2 -n)) = c$. $n^2 < c < (n+1)^2\implies 0< c - n^2<(n+1)^2 -n^2 = (n+1 +n)(n+1 -n) = 2n+1 \implies 0\le c-n^2 < 2n + 1\implies c-n^2-n < n + 1\implies c-n^2 -n \le n$.
So $g(c) = (\lfloor \sqrt{c} \rfloor, c - \lfloor \sqrt{c} \rfloor^2 - \lfloor \sqrt{c} \rfloor)$ will do. But this requires $ c - \lfloor \sqrt{c} \rfloor^2 - \lfloor \sqrt{c} \rfloor)\in \mathbb N$. i.e $ c - \lfloor \sqrt{c} \rfloor^2 > \lfloor \sqrt{c} \rfloor) $
Other wise we must solve for $f(a,b) = a+b^2=c; a< b$ first.
If $n^2 \le c < (n+1)^2$ then $f(c-n^2, n) = c$ if $c-n^2 < n$. As $n^2 \le c < n^2 + 2n+1$ we know $0\le c-n^2 < 2n+1$.
So $h(c) = (c - \lfloor \sqrt{c} \rfloor^2 , \lfloor \sqrt{c} \rfloor)$ will do if $c - \lfloor \sqrt{c} \rfloor^2>0$ which only fails if $n^2 \le c < n^2 + 1$ . If this is the case then $c - \lfloor \sqrt{c} \rfloor^2 < 1 \le \lfloor \sqrt{c} \rfloor$.