Let's take a look at this simple task (Pythagorean Triples):
Calculate $A$ and $B$ such that $A^2 + B^2 = C^2$.
$C$ is given.
Is there any way to find an upper bound for $A$, $B$, $A^2$, and $B^2$?
The upper bound will be a function of $C$.
$0 < A < f_1(C)$
$0 < B < f_1(C)$
$0 < A^2 < f_2(C)$
$0 < B^2 < f_2(C)$
$f_1(C) = ?$
$f_2(C) = ?$
Any ideas?
On
There are an infinite number of pythagorean triples for which $B + 1 = C$. Let $A = 2n+1$. Let $B = 2n^2 + 2n$ and $C = B + 1 = 2n^2 + 2n + 1$.
$$\begin{align} \\ A^2 + B^2 &= (2n+1)^2 + (2n^2 + 2n)^2 \\ & = (4n^2 + 4n + 1) + (4n^4 + 8n^3 + 4n^2) \\ & = 4n^4 + 8n^3 + 8n^2 + 4n + 1 \\ & = (2n^2 + 2n + 1)^2 \\ & = C^2 \end{align} $$
On
Using Euclid's formula $\quad A=m^2-n^2\quad B=2mn\quad C=m^2+n^2\quad$ if you are given $C$, you can find $A$ and $B$ directly, if they exist, by first solving the $C$-function for $n$. Doing so suggests limits or boundaries for $m$-values that may generate integer values for $n$.
$$C=m^2+n^2\Rightarrow n=\sqrt{C-m^2}\qquad\text{where}\qquad \biggl\lceil\sqrt{\frac{C}{2}}\biggr\rceil \le m < \sqrt{C}$$
The lower limit ensures that $m-n \ge 1$ and the upper limit ensures that $n\in\mathbb{N}$. The most common example in recent usage is one for which there are four sets of triples: $$C=1105\qquad\text{where}\qquad \biggl\lceil\sqrt{\frac{1105}{2}}\biggr\rceil=24 \le m < \sqrt{1105}=33$$
In this range, we find $\quad m\in\{24,31,32,33\}\quad \implies\quad n\in\{23,12,9,4\}\quad$ meaning $$F(24,23)=(47,1104,1105)$$ $$F(31,12)=(817,744,1105)$$ $$F(32,9)=(943,576,1105)$$ $$F(33,4)=(1073,264,1105)$$
As far as finding lower and upper bounds, about as far as we can say about that is as seen in examples below$$\quad 3\le A \le C-2\qquad\text{ and }\qquad 4\le B \le C-1$$
$$\text{A:}\qquad (3,4,5)\qquad (249999,1000,250001)\qquad (999999999999,2000000,1000000000001)$$ $$\text{B:}\qquad (3,4,5)\qquad (1001,501000,501001)\qquad (1999999,1999998000000,1999998000001)$$
Note that $3,4,5$ is a triple and $n^2-1,2n, n^2+1$ is also a triple.
Your upperbound must satisfy $$f(n^2+1) > n^2-1$$
So if you are looking for a "nice function" (Polynomial), the best upperbound must satisfy $f(C) \geq c-2$ infinitelly often, and you can make it $C-1$. To replace $C-2$ by $C-1$ you just observe that $k,n,n+1$ is a solution whenever $k^2=2n+1$, that is whenever $n=\frac{m^2-1}{2}$ for $m$ an odd integer.
Moreover, these functions yield $\leq$ not strict inequalities. If you want strict inequalities, there is no best polynomial upperbound. You might want your function to only take integer values.
If by function you mean any function, then the answer is simple: your problem is the definition of your function. And again, since all your variables are integers, there is no best upperbound to satisfy your strict inequalities, unless again you restrict the codomain to integers.