Let's say we have a volume-based fee structure that takes a 5% fee of a transaction from \$0, and a 2% fee at \$1000, with the intermediate amount being a linear interpolation of these two points - for example, \$500 would equate to a 3.5% fee.
The crux of the question comes when we want to calculate the inverse. I.e. instead of "if I give \$250, what do I receive?", we need to answer "if I need exactly \$250, what do I need to give?".
Given the linear interpolation of the fees, simply taking the inverse of the amount is no longer possible, as we would end up in a slightly more favorable fee rate.
An example of what I've tried so far, which due to the above doesn't quite work:
$$feeRate = 5 - (5 - 2) * \frac{250}{1000} = 4.25\%$$ $$amountOut = 250 * (1 - \frac{4.25}{100}) = \$239.375$$
Calculating the inverse:
$$feeRate = 5 - (5 - 2) * \frac{239.375}{1000} = 4.281875\%$$ $$amountOut = 239.375 * (\frac{1}{1 - 4.281875 / 100}) \approx \$250.083$$
Is there any way, creative or not, to make sure this inverse way of calculating the fee is possible?
I've also tried "chunking" the linear interpolation, i.e. making it into a series of steps instead of completely linear, but that just made it work for the amounts where both fell onto the same steps, but made it even less accurate when they didn't.
As long as any kind of solution is programmable, I'd be satisfied, even if it requires some sort of multi-step process/brute forcing. Herein also if rounding the fee percent to, for example, 2 decimals makes it easier, that wouldn't be a problem either.
I don't really understand why, in order to do the inverse, you still started from the interpolated rate. If you know that you end up with $239.375$, it seems to me that you just need to invert the second equation you wrote.
Plus, if you plug that number into the interploated rate, you are answering to your first question ("if I give you $239.375$, what do I receive?") with the new amount $239.375$.
Defining $A$ as what you called amountOut, $r$ as the feeRate and $x$ the amount you need we have:
$$ A = x \; (1-r) \Rightarrow r = 1- \frac{A}{x} $$
In your example $A=239.375, \; x = 250 \;$ which gives the desired $r=4.25\%$