Much time has passed since I studied any form of math so I wanted to take this cheap shot of asking someone else to think for me.
I need to write some software that, for any given real number argument y between 0.0 and 510.0 (approximated by a float) finds two natural numbers h and m (expressed as integers between 0-255) that minimise the difference between y and h * m / 128.
I think that the following code is a brute force solver of the problem but it is excruciatingly inefficient:
function errFunc (y, h, m)
{
return Math.abs (y - (h * m / 128.0));
}
function fit (y)
{
var minError = 2; // y <= 510, h <= 255, h <= 255. So errFunc(y) < 2
var h;
var m;
for (h = 0; h < 256; h++)
{
for (m = 0; m < 256; m++)
{
e = errFunc (y, h, m);
if (e < minError)
{
minError = e;
optimalH = h;
optimalM = m;
}
}
}
}
Any help would be appreciated.
Instead of trying all combinations of $h$ and $m$ values you could loop through all the possible $h$ and try just $m=\left\lfloor \frac{128y}{h} +\frac12 \right\rfloor$ for each of them.
You'll need to special-case $h=0$, but zeroes will be involved if and only if $y\le\frac1{256}$, so you can handle that once and for all at the beginning.
You can optimize further by trying only $h$ in the interval $1$ to $\Bigl\lfloor \sqrt{128y}\Bigr\rfloor$.