Course: Numerical Methods

SOURCES OF ERROR

Effect of Carrying Significant Digits

By Autar Kaw

Learn the effect of carrying  limited number of significant digits for a real-life problem.  We regressed the data of coefficient of thermal expansion of steel as a function of temperature to a second order polynomial using Excel.  We found that the general format of the regression model provided by Excel creates large round-off errors because of carrying limited significant digits.  A solution to this issue is also presented.

Round-off Error: Definition and Examples [YOUTUBE 3:47]

Effect of Carrying Significant Digits [YOUTUBE 7:48]

Truncation Error: Definition [YOUTUBE 8:34]

Truncation Error: Example: Series [YOUTUBE 6:44]

Truncation Error: Example: Integration [YOUTUBE 6:12]

Truncation Error: Example: Differentiation [YOUTUBE 10:14]

Copyrights: University of South Florida, 4202 E Fowler Ave, Tampa, FL 33620-5350. All Rights Reserved. Questions, suggestions or comments, contact kaw@eng.usf.edu  This material is based upon work supported by the National Science Foundation under Grant# 0126793, 0341468, 0717624,  0836981.  Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.  Other sponsors include Maple, MathCAD, USF, FAMU and MSOE.  Numerical Methods for Undergraduates by http://numericalmethods.eng.usf.edu is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.  Based on a work at numericalmethods.eng.usf.edu.