I'm self-learning LA through an online book. And wanted to clear up my confusion regarding the Rounding Function.
I'm finding it hard to describe the problem, so I first started off by describing how this book defines the rounding function for floating-point numbers, then a few obvious examples, then finally my doubts with how this function would result for $x=0.21045$, or any number outside the given precision($p$) value.
The rounding function is defined as:
For any number within the set of floating point numbers $$\mathbb{F}_{p,b,N} =\pm0.d_1d_2\dots d_p\times b^n $$where $p$ is the precision, $b$ is the base, and $n$ is the exponent within the range $\{-N\leq n\leq N\}$.
A couple of examples are given as follows for the values that are not in $\mathbb{F}_{3,10,3}$ but rounded off to a number within the set: \begin{align} x = 0.2103 × 10^3 &\rightarrow 0.210\\ x = 0.21051 × 10^3 &\rightarrow 0.211\\ x = 0.21037 × 10^3 &\rightarrow 0.210 \end{align}
Third example is maybe done as follows: \begin{align} 0.21037 \\ 0.2103\tag{since $p=3$, we ignore all numbers after $d_{3+1}$ i.e 3}\\ 0.210\tag*{$\because d_{p+1}=3<5$} \end{align}
Here are my doubts:
I came up with a number similar to the examples above, $x=0.21045$, intuitively its round-off should be $x=0.211$ but by the same methodology as the above example we get $x=0.210$.
\begin{align}
0.21045 \\
0.2104\tag{since $p=3$, we ignore all numbers after $d_{3+1}$ i.e 4}\\
0.210\tag*{$\because d_{p+1}=4<5$}
\end{align}
I am not sure what exactly is wrong here, I think there may be a problem with
- either my methodology of implementing the rounding function: ignoring all the values after a certain radix point.
- or with the assumption that the answer should be $0.211$ when in fact within the set $\mathbb{F}_{3,10,3}$ the output $0.210$ is correct.
The definition says you just look at the first digit past what you want to round to. If it is $5$ or greater you increase the last digit you want to keep by $1$. If it is $0$ to $4$ you leave the last digit the same. In your example of $21037$ being rounded to three places, the first digit to remove is $3$, so you round down. You do not even look at the $7$. Similarly, in $21045$ the first digit you want to remove is $4$, which is less than $5$. You don't even look at the $5$. You also round this to $210$. This makes sense because you are always picking the closer value unless the part to remove is precisely $0.50000\ldots$. In that case there is no closer value. Either way you round is the same distance away.
When we round fixed point numbers we worry about an upward bias if we round all the exactly $0.5$ values up to $1$. Often we round exactly $0.5$ to the even number. For real numbers getting exactly $0.5$ is not supposed to happen or not enough to matter.