floating point
Scientific notation computing
Floating point is a method of representing numbers in computing where the decimal point can "float" to different positions, allowing representation of very large or very small values.
The term contrasts with fixed point, where the decimal stays in one place.
Floating point arithmetic involves trade-offs: it handles extreme ranges but can introduce tiny rounding errors.
Floating point precision issues caused the calculation to fail.
Floating-point arithmetic standard
- mantissaLogarithm's fractional component
- exponentSuperscript showing power
- binaryConsisting of two parts
- hexadecimalSixteen-digit numeral system
scientific notation · normalized · subnormal · encoding · format
Joining quantities mathematically
- subtractionRemoving quantities mathematically
- multiplicationProduct-finding operation
- divisionSplitting into parts
- roundingAdjusting to simpler values
truncation · overflow · underflow · commutativity · operand · result
Degree of correctness
- significant digitsNumerals showing measurement precision
- epsilonGreek fifth letter
- roundoffNumerical simplification technique
- doubleTwice the amount
single · half · extended · decimal places · relative error
Twice the amount
- NaNUnrepresentable numerical value
- infinityUnlimited extent or quantity
- comparisonAssessment of similarities
- castingShaping through liquid form
Supplementary computational unit
- ALUProcessor calculation component
- registerRecord officially; vocal range
- SSEParallel data instruction set
- GPUVisual processing hardware
processor · accelerator · pipeline · latency · throughput · instruction set
IEEE 754
Floating-point arithmetic standard
- mantissaLogarithm's fractional component
- exponentSuperscript showing power
- binaryConsisting of two parts
hexadecimal · scientific notation · normalized · subnormal · encoding · format
addition
Joining quantities mathematically
- subtractionRemoving quantities mathematically
- multiplicationProduct-finding operation
- divisionSplitting into parts
rounding · truncation · overflow · underflow · commutativity · operand · result
accuracy
Degree of correctness
- significant digitsNumerals showing measurement precision
- epsilonGreek fifth letter
- roundoffNumerical simplification technique
double · single · half · extended · decimal places · relative error
double
Twice the amount
- NaNUnrepresentable numerical value
- infinityUnlimited extent or quantity
- comparisonAssessment of similarities
casting · literals · serialization · parsing
coprocessor
Supplementary computational unit
- ALUProcessor calculation component
- registerRecord officially; vocal range
- SSEParallel data instruction set
GPU · processor · accelerator · pipeline · latency · throughput · instruction set
- computingcomputer, computers, software, hardware, code
- math operationsaddition, adding, subtraction, subtracting, multiplication
- notationnotation, musical notation, score, sheet music, tablature
- number formatsinteger, decimal, binary, percentage, hexadecimal
- number formsnumerals, digits, digit, integers, integer
- computational mathalgorithm, linear algebra, calculus, differential equations, finite element