There is a number circle numbered from 0..N and have 2 points in the circle, where I need to find the middle point between 2 points on the circle where the the middle point will be on the circle and the distance should be minimal. In the below image the answer is between N-1 and 0 = N-0.5 (as integer it's N-1).
Is there any formula or algorithm to find the middle point? (I need the answer to solve a programming/algorithm problem)

I believe you can calculate $m = \dfrac{(a+b)}{2}$ to get one of the midpoints, then add $\dfrac{N}{2}$ and mod $N$ again to get the other solution, then you can just check which solution is closer.
Edit: correction from comments