
Maine DEP Monthly
Maine and WEF's
Penobscot Watershed and Development of a TMDL
Maine Rural Water
Maine
Wastewater Control Association 
Rounding Numbers Tim Loftus The analysis is complete, your technique was good, there were no interferences, and your equipment was operating at peak efficiency. How accurate is the result? That depends on how accurate your glassware and instrumentation is and on the purity of the chemicals used. The final result cannot be more accurate than the least accurate part of the analysis. Therefore, how you record a result should reflect the accuracy you have achieved. For example, a total phosphorus value of 5.896 mg/L suggests that you have achieved a high degree of accuracy, but it’s also something that should raise a red flag for those who read it. This level of accuracy is generally unachievable because much of the laboratory equipment, especially glassware, does not have that level of accuracy. Again, the result cannot be more accurate than the least accurate part of the analysis. This means that you will need to round numbers to reflect the accuracy of the result. The most common method for rounding numbers is as follows: Drop all digits in a number that are not significant. (See previous article “Significant Digits.”) If the number immediately to the right of the significant numbers is between 6 and 9, round up one unit. For example, if you wish to round the number 53.68 to three significant digits, round the 6 digit to 7 since the nonsignificant digit 8 is between 6 and 9. The result is 53.7. To round 53.68 to two significant digits, round the 3 to 4 since the nonsignificant digit immediately to the right is 6. The resulting number is 54. If the number immediately to the right of the significant numbers is between 0 and 4, leave the significant numbers unchanged and drop the nonsignificant numbers. For example, 53.64 rounded to three significant digits is 53.6. If the number immediately to the right of the significant numbers is 5, round to the nearest even number. For instance, 53.65 rounded to three digits is 53.6. For 53.75, the rounded number becomes 53.8. When performing a series of calculations, round the final result rather than the intermediary numbers. For example, the calculations for BOD use several numbers, all having different amounts of significant digits. Perform the calculations, then round the result. For a real life example using the BOD equation, (DO changeseed correction)x dilution = BOD value, and inserting typical numbers we have: ((8.95 mg/L  5.25 mg/L)  (0.6952 mg/L)) x (300 mL/6.0 mL) = 150.24 mg/L. Since the number with the least amount of significant digits used in this calculation dictates how many significant digits should be in the final answer, round the final answer to two significant digits. The rounded result is 150 mg/L. If you incorrectly rounded each of the numbers in the calculation to two significant digits, you would get: ((9.0  5.2)  (0.70)) x ((300/6.0) = 155. When the answer is rounded to two significant digits, the result is 160 mg/L. The correct result is 150 mg/L, the incorrect result is 160. As you can see, there is quite a difference. And this difference may put you in violation of your discharge permit or place your analysis outside acceptable QA/QC ranges. As usual, there are a few exceptions. For a complete explanation of these exceptions, refer to Standard Methods for the Examination of Water and Wastewater or Wastewater Sampling for Process and Quality Control MOP OM1. The information in this article is very general. As usual, check your federal, state, and local regulations. You may have additional regulations or requirements that you must meet. If you have any questions, suggestions, or comments, please contact NEWEA Lab Practices Committee Chair Phyllis Arnold Rand 2077820917 (prand@lawpca.org) or Tim Loftus at (508) 9493865 (timloftus@msn.com).

