Precision (computer science) - Wikipedia, the free encyclopedia

1 of 1

https://en.wikipedia.org/wiki/Precision_(computer_science)

From Wikipedia, the free encyclopedia

In computer science, the precision of a numerical quantity is a measure of the detail in which the
quantity is expressed. This is usually measured in bits, but sometimes in decimal digits. It is related to
precision in mathematics, which describes the number of digits that are used to express a value.

Precision is often the source of rounding errors in computation. The number of bits used to store a
number will often cause some loss of accuracy. An example would be to store "sin(0.1)" in IEEE
single precision floating point standard. The error is then often magnified as subsequent
computations are made using the data (although it can also be reduced).

Arbitrary-precision arithmetic
Extended precision
IEEE754 (IEEE floating point standard)
Integer (computer science)
Significant figures
Truncation
Retrieved from "https://en.wikipedia.org/w/index.php?title=Precision_(computer_science)&
oldid=708962480"
Categories: Computer data
This page was last modified on 8 March 2016, at 12:30.
Text is available under the Creative Commons Attribution-ShareAlike License; additional terms
may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia is a
registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.

2016/4/26 12:18