Floating point approximation to logarithms by powers and bit counting?

Question

Earlier I both asked questions and found some interesting sources on the internet of how to do approximate division by combining and counting number of $0$s following most significant bit in denominator.

Example: We represent a fraction $7/3$ as two ordered integers $(7,3)$.

We want to calculate a floating point approximation of it.

If we multiply both numerator and denominator by 11 we get the representation $(77,33)$ (for the same fraction).

Looking at the representation of 33 in binary: 100001, we can be tempted to replace the division by a bit-shift, if we pretend the binary expansion is 100000 ( pretending the second 1 and all that follows it is a 0 ).

The relative error will be about $1/32$ or in other words $3\%$. But all involved operations are super-fast to perform in electronics hardware (as long as we know a binary multiplicative "number-friend" of 3 is 11).

Now to my question, can we apply this same idea to approximate logarithms?

For example to calculate the 2-logarithm of $3$, we calculate $3\cdot 3$ is 9, bit expansion 1001, which we pretend to be 1000, giving an approximation of $\log_2(9) \approx 3$ or with log law: $2\log_2(3) \approx 3$ and therefore $\log_2{3} \approx \frac{3}{2} = 1.5$. The real value is $1.585...$, somewhat close(?)

Given that only multiplications and bit shifts are required and that we can without trouble throw away the least significant half of the product result, could this be a feasible approach for a floating point log approximator?

I suspect that it is reasonable to believe that this has already been tried, so any references you may have to such methods are heartily welcomed.

Answer 1

Thanks for all the input. The algorithm I thought about was a bit more like this

1. take the number $a$ which we want to calc the log of. set $b=a$
2. loop until $b$ has $k$ most significant $1$ bits in a row:
   1. $b = a\times b$
   2. shift $b$ some bits so that we are sure to avoid overflow in subsequent multiplications, but count how large shift and store in exponent.
3. the position of the last $0$ before the first $1$ marks the exponent of $a^n$, where $n$ is the number of loop iterations
4. return $e/n$ where e is the sum of position in 3. and the accumulated exponent in the loop.

( Note that if we have floating point arithmetics we can skip keeping track of the exponent. Also the division by $n$ will be quickly implementable as a floating point multiplication by $\frac{1}{n}$ which we can tabulate if we would want )

We try on the number 666 with an implementation in the c language with gcc as compiler. We provide the value from c library logf function for reference.

$$\left[\begin{array}{cccc}n&k&\text{output}&\text{logf}(666.0)\\2& 1&9.500000&9.379378\\5& 2&9.400001&9.379378\\13& 3&9.384616&9.379378\\29& 4&9.379311&9.379378\\\end{array}\right]$$

As we can see the number of iterations required to first find $k$ ones in a row grows a bit quickly. We would want to get an estimate faster, but as a first prototype it sure works.

Answer 2

To use this approach you need a power of $3$, not a multiple, to be close to a power of $2$, and then you need to find it. If you want $\log_2 5$ you could note that $125=5^3 \approx 2^7=128$ and say $3 \log_2 5 \approx 7, \log_2 5 \approx \frac 73 \approx 2.3333$ while the real value is about $2.322,$ so it is rather accurate. It is not obvious to me how you find them. Say you wanted $\log_2 19.$ How would you find a power of $19$ that is close to a power of $2$? It turns out that $19^4=130321$ is quite close to $2^{17}=131072$, but I found it by computing $\log_2 19\approx 4.2479$ and rouding that to $\frac {17}4$

Answer 3

The details of the proposed algorithm are not entirely clear to me, but it seems very similar to what is described in the Wikipedia entry for binary logarithm in the section "Iterative approximation". Some references describing algorithms of this kind are:

J. C. Majithia and D. Levan, "A note on base-2 logarithm computations". Proceedings of the IEEE, Vol. 61, No. 10, October 1973, pp. 1519-1520 (Reprinted in Earl E. Schwartzlander (ed.), Computer Arithmetic Volume 1, IEEE Computer Society Press, 1990, pp. 231-232)

Alan H. Karp, "Exponential and Logarithm by Sequential Squaring". IEEE Transactions on Computers, Vol. 33, No. 5, May 1984, pp. 462-464

Hao-Yung Lo and Jau-Ling Chen, "Hardwired Generalized Algorithm for Generating the Logarithm Base-$k$ by Iteration". IEEE Transactions on Computers, Vol. 36, No. 11, November 1987, pp. 1363-1367

Clay S. Turner, "A Fast Binary Logarithm Algorithm". IEEE Signal Processing Magazine, Vol. 27, No. 5, September 2010, p. 124,140.

While the approach is certainly feasible, the preferred method for computing logarithms in floating-point arithmetic at this time is typically polynomial minimax approximation. See this answer for a worked example. For fixed-point computation on simple processors, the bit-wise computation by a pseudo-division process is often attractive; it goes back to the logarithm computations of H. Briggs in the early 17^th century. See this answer for a worked example.