MATH
Login Or Sign up

List Question

15
115
Views

Approach to address errors from float precision limits by its storage limit

Published on
#floating-point
671
Views

Computing continued fraction expansions

Published on
#numerical-methods #continued-fractions #floating-point
151
Views

Floating point division resulting in a value exceeding 1 but should be equal to 1...maybe!

Published on
#numerical-methods #floating-point
43
Views

Using the rounding function to compute $x=0.21045$ should the output be $0.210$ or $0.211$ within a certain set of floating points?

Published on
#floating-point #rounding-unit
310
Views

Cancellation in the exponential function

Published on
#numerical-methods #computational-mathematics #floating-point
82
Views

Numerical method to solve difference equation

Published on
#numerical-methods #recurrence-relations #computer-science #floating-point
329
Views

Avoiding catastrophic cancellation in $1-\operatorname{sinc}x$

Published on
#numerical-methods #floating-point #machine-precision #catastrophic-cancellation
490
Views

Is there an icon latin, glyph or symbol that means rounding?

Published on
#mathematical-modeling #floating-point #rounding-unit
576
Views

Complex floating point types

Published on
#reference-request #complex-numbers #numerical-methods #computational-mathematics #floating-point
121
Views

What is the ULP variance of the common implementation of lerp?

Published on
#interpolation #floating-point
213
Views

Relative error when chopping is used of a 32 bit floating point number in IEEE 754 format

Published on
#numerical-methods #notation #numerical-linear-algebra #floating-point
103
Views

Floating point arithmetic error propagation

Published on
#numerical-methods #floating-point #error-propagation
298
Views

In a floating-point system, is the unit roundoff $\epsilon_{mach}$ necessarily a machine number?

Published on
#numerical-methods #computational-mathematics #number-systems #floating-point #scientific-notation
128
Views

Limit of Euler's number failing due to precision errors - A surprising case. Why does it happen?

Published on
#limits #exponential-function #floating-point #eulers-number-e #machine-precision
43
Views

Sequence in floating point arithmetic.

Published on
#calculus #numerical-methods #numerical-calculus #floating-point

Trending Questions

Popular # Hahtags

second-order-logic numerical-methods puzzle logic probability number-theory winding-number real-analysis integration calculus complex-analysis sequences-and-series proof-writing set-theory functions homotopy-theory elementary-number-theory ordinary-differential-equations circles derivatives game-theory definite-integrals elementary-set-theory limits multivariable-calculus geometry algebraic-number-theory proof-verification partial-derivative algebra-precalculus

Popular Questions

Copyright © 2021 JogjaFile Inc.