If you ask a programming language to calculate 0.6 + 0.7, you’ll get something like 1.2999998, and that’s because of how floating point numbers are represented in computers. But if you ask a calculator, or even the calculator app on your phone, to do the same thing, it will give you the correct answer: 1.3. Why? Do calculators not represent floats the same way that other computers do?
2026-03-26 12:33:06.1774528386
How do calculators represent floating points (somewhat) perfectly?
119 Views Asked by Bumbble Comm https://math.techqa.club/user/bumbble-comm/detail At
1
There are 1 best solutions below
Related Questions in COMPUTER-SCIENCE
- What is (mathematically) minimal computer architecture to run any software
- Simultaneously multiple copies of each of a set of substrings of a string.
- Ackermann Function for $(2,n)$
- Algorithm for diophantine equation
- transforming sigma notation into harmonic series. CLRS A.1-2
- Show that if f(n) is O(g(n) and d(n) is O(h(n)), then f(n) + d(n) is O(g(n) + h(n))
- Show that $2^{n+1}$ is $O(2^n)$
- If true, prove (01+0)*0 = 0(10+0)*, else provide a counter example.
- Minimum number of edges that have to be removed in a graph to make it acyclic
- Mathematics for Computer Science, Problem 2.6. WOP
Related Questions in FLOATING-POINT
- The upper and lower limits of IEEE-754 standard
- How to convert $\ln x - \ln y$ into a more accurate floating point representation?
- Writing A Number In Floating Point With $5$ Significant Digits
- Understanding the amount of FLOPS required to perform a single iteration in GCR
- Conditioning of the linear systems in the inverse or Rayleigh quotient iteration algorithms
- Distinguising two error terms in rounding error
- Understanding Floating point arithmetic
- Why floating point relative accuracy is needed?
- What is the correct way to round 331.449999 to 1 decimal place
- What does the floating point arithmetic contribute to the rounding error in Mathematica?
Related Questions in CALCULATOR
- Why does a calculator say that 3% + 3% = .0309?
- Calculators doing numeric integration
- why does $\sin\left(\sin x\ +\ \cos y\right)=\cos\left(\sin\left(x\cdot y\right)+\ \cos\ x\right)$ look so weird?
- Physical Calculator with Hyperreals and Multiple Dimensions?
- Evaluate boolean algebra expression for all binary numbers on a calculator
- Solving equations by iteration (Calculator ANS key)
- ti-89 is not showing a proper graph!
- How to convert 5.5271479e-74 to realistic number
- Ti84 Plus "$a+bi$" mode not working
- How to solve this trigonometric equation using Casio CFX 9860 series?
Trending Questions
- Induction on the number of equations
- How to convince a math teacher of this simple and obvious fact?
- Find $E[XY|Y+Z=1 ]$
- Refuting the Anti-Cantor Cranks
- What are imaginary numbers?
- Determine the adjoint of $\tilde Q(x)$ for $\tilde Q(x)u:=(Qu)(x)$ where $Q:U→L^2(Ω,ℝ^d$ is a Hilbert-Schmidt operator and $U$ is a Hilbert space
- Why does this innovative method of subtraction from a third grader always work?
- How do we know that the number $1$ is not equal to the number $-1$?
- What are the Implications of having VΩ as a model for a theory?
- Defining a Galois Field based on primitive element versus polynomial?
- Can't find the relationship between two columns of numbers. Please Help
- Is computer science a branch of mathematics?
- Is there a bijection of $\mathbb{R}^n$ with itself such that the forward map is connected but the inverse is not?
- Identification of a quadrilateral as a trapezoid, rectangle, or square
- Generator of inertia group in function field extension
Popular # Hahtags
second-order-logic
numerical-methods
puzzle
logic
probability
number-theory
winding-number
real-analysis
integration
calculus
complex-analysis
sequences-and-series
proof-writing
set-theory
functions
homotopy-theory
elementary-number-theory
ordinary-differential-equations
circles
derivatives
game-theory
definite-integrals
elementary-set-theory
limits
multivariable-calculus
geometry
algebraic-number-theory
proof-verification
partial-derivative
algebra-precalculus
Popular Questions
- What is the integral of 1/x?
- How many squares actually ARE in this picture? Is this a trick question with no right answer?
- Is a matrix multiplied with its transpose something special?
- What is the difference between independent and mutually exclusive events?
- Visually stunning math concepts which are easy to explain
- taylor series of $\ln(1+x)$?
- How to tell if a set of vectors spans a space?
- Calculus question taking derivative to find horizontal tangent line
- How to determine if a function is one-to-one?
- Determine if vectors are linearly independent
- What does it mean to have a determinant equal to zero?
- Is this Batman equation for real?
- How to find perpendicular vector to another vector?
- How to find mean and median from histogram
- How many sides does a circle have?
The short version is, they're cheating. Calculators or calculator apps are, if anything, less capable than computers and programming languages in general. That said, they're also specialized for doing exactly that sort of thing, so they're more inclined to take 12.9999998 and round it for you to 13. Programming languages will do the same thing, but depending on which language and how precise the number, you might have to ask them to.
Take any calculator and try something like adding $1+10^{-100}-1$, and in my experience, you'll get $0$, not $0.0000\dots 001$.