On various kinds of rounding rules for multiplication and division

Wei Da Chen*, Wei Lee, Christopher L. Mulliss

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

A detailed investigation of three different rounding rules for multiplication and division is presented, including statistical analyses via Monte-Carlo simulations as well as a mathematical derivation. This work expands upon a previous study by Mulliss and Lee (1998), by making the more realistic assumption that the contributing uncertainties are statistically independent. With this assumption, it is shown that the so-called standard rounding rule fails over 60% of the time, leading to a loss in precision. Two alternative rules are studied, and both are found to be significantly more accurate than the standard rule. One alternative rule requires one extra significant digit beyond that predicted by the standard rule. The other requires one to count numbers whose leading digit is 5 or greater as having an extra significant digit, and then to apply the standard rule. Although the second alternative rule is slightly more accurate, the first is shown to be completely safe for data - never leading to a truncation of digits that contain significant information. Accordingly, we recommend the first alternative rule as the new standard.

Original languageEnglish
Pages (from-to)335-346
Number of pages12
JournalChinese Journal of Physics
Volume42
Issue number4 I
StatePublished - 1 Aug 2004

Fingerprint Dive into the research topics of 'On various kinds of rounding rules for multiplication and division'. Together they form a unique fingerprint.

Cite this