I have sets of points. E.g.: 5.664, 2.292, 1.368, 0.18, 3.3, 4.74, 7.812, 6.564, 5.352, 4.008, 2.568, 5.352
I'd like to shift them a bit (add some uniform dx to all of them) to make them closer to the nearest int.
In the above example, the sum of squares of distances from nearest int is ~1.2.
I guess I'm trying to minimize this sum of squares... adding dx=0.55 to all the numbers reduces that sum to ~0.63.
The question is, is there a proper way to get to dx=0.55, other than trial and error?
Just to give some scope - these points are coordinates, in pixels, and shifting them closer to nearest int improves anti-aliasing so drawings look better.
Thanks!
Background
If we consider how the sum of the squares of just one value changes as you change $dx$ you get a graph like:
Note there is a discontinuity. Once you combine multiple graphs then discontinuities become less noticible but are still present.
(Note the black line here is the average rather than the sum of the squares of the distance to the nearest int.)
You will observe that when there are multiple high points close together they correspond to higher point in the average and when there are multiple low points close together they correspond to lower point in the average.
Taking this over all the points you will get that the lowest point will be at the average of all the lowest points. You can calculate the individual low points by looking at how far each point is from its nearest integer. So your process/code should do the following:
Process to find Minimum $dx$
Example on your values
$$0.664, 0.292, 0.368, 0.18, 0.3, 0.74, 0.812, 0.564, 0.352, 0.008, 0.568, 0.352$$
$$\left(0.664+0.292+0.368+0.18+0.3+0.74+0.812+0.564+0.352+0.008+0.568+0.352\right)\div12=5/2\div12=0.43333\cdots$$
$$dx=1-0.43333\cdots=0.56666\cdots$$
This will give you a sum of distance to nearest integer of $0.626186667\cdots$