If you are applying a discount to a price and you have two possible choices on the discount to apply, a percentage or a fixed amount, e.g. I could remove a fixed amount of 5 or I can remove a 20% of the final price HOWEVER at this point we are unaware of the final price.
As you are unable to determine the final price the discount may be greater in some situations than other, e.g. if the price is 20 and a 20% discount would make your total 18 whereas applying a discount of 5 would make it 15, 5 greater than 20% however if the price is 100 a 20% discount would make your final price 80 whereas a discount of 5 would make it 95 making 20% more valuable.
How can I determine which one will yield a greater discount (e.g. the final price will be lower if x is applied over y)?
The two discount schemes will give the same result for an original price $p$ where $$p - \frac{20}{100}p = p - 5$$ which is not difficult to solve for $p$.
As you say, the left hand side will be smaller (larger) than the right hand side when $p$ is larger (smaller) than that solution.