
- Finite-precision arithmetic can break algebraic identities because truncating recurring expansions (like a stored 1/3) removes the infinite tail, so adding three truncated thirds yields slightly less than one (e.g., 0.999999...).@technology· Software
Finite-precision arithmetic can break algebraic identities because truncating recurring expansions (like a stored 1/3) removes the infinite tail, so adding three truncated thirds yields slightly less than one (e.g., 0.999999...).
- Treating floating point as base-2 scientific notation makes its behavior intuitive because it's implemented as a fixed-length significand plus an exponent, so thinking in halves, quarters, and eighths explains rounding and significant-figure limits.@technology· Software
Treating floating point as base-2 scientific notation makes its behavior intuitive because it's implemented as a fixed-length significand plus an exponent, so thinking in halves, quarters, and eighths explains rounding and significant-figure limits.
- Binary floating-point is a poor choice for currency because it can't exactly represent many decimal amounts (like tenths), which causes visible rounding errors—use a decimal type or store values as integer cents instead.@technology· Software
Binary floating-point is a poor choice for currency because it can't exactly represent many decimal amounts (like tenths), which causes visible rounding errors—use a decimal type or store values as integer cents instead.
- Floating-point precision limits are usually acceptable for practical tasks because the absolute rounding error is typically far smaller than real-world tolerances (for example, a sub-pixel difference in screen coordinates), so results remain effectively correct.@technology· Software
Floating-point precision limits are usually acceptable for practical tasks because the absolute rounding error is typically far smaller than real-world tolerances (for example, a sub-pixel difference in screen coordinates), so results remain effectively correct.
- Many base-10 fractions cannot be represented exactly in binary because converting them produces recurring binary digits (e.g., 0.1 → 0.000110011...), and floating-point formats only store a finite number of bits so the repeating tail is cut off.@technology· Software
Many base-10 fractions cannot be represented exactly in binary because converting them produces recurring binary digits (e.g., 0.1 → 0.000110011...), and floating-point formats only store a finite number of bits so the repeating tail is cut off.
- Floating-point arithmetic is fast and efficient because it stores values as a mantissa plus an exponent (like scientific notation), letting hardware represent extremely large and tiny numbers compactly and perform optimized arithmetic operations.@technology· Software
Floating-point arithmetic is fast and efficient because it stores values as a mantissa plus an exponent (like scientific notation), letting hardware represent extremely large and tiny numbers compactly and perform optimized arithmetic operations.
- Simple decimal sums like 0.1 + 0.2 can produce unexpected results because computers store numbers in binary scientific notation with limited significant bits, so decimal fractions become recurring binary expansions that are truncated and produce tiny rounding errors.@technology· Software
Simple decimal sums like 0.1 + 0.2 can produce unexpected results because computers store numbers in binary scientific notation with limited significant bits, so decimal fractions become recurring binary expansions that are truncated and produce tiny rounding errors.
Want more like this?
Every card on Korva is an insight someone saved from a podcast or video they loved.