Nonlinear Regression EBook

On non Internet Explorer browsers, the equations may not show up.

As an alternative or if you prefer,

you can see the pdf version of the ebook instead

Title	An interactive e-book for illustrating linear regression
Creator	Autar K Kaw
Subject and Keywords	Nonlinear Regression, Regression, Mathcad, Maple, Mathematica, Matlab, Simulations.
Description	This is an interactive E-book for illustrating linear regression. It includes links to examples, simulations in Mathcad, Maple, Mathematica, and Matlab for the algorithm, and a PowerPoint presentation.
Publisher	Holistic Numerical Methods Institute, College of Engineering, University of South Florida, Tampa, FL 33620-5350.
Contributors	Autar Kaw, Egwu Kalu
Format	Text/HTML
Last Revised	October 3, 2007
Identifier	http://numericalmethods.eng.usf.edu/ebooks/straightline_06reg_ebook.pdf
Language	English
Rights	http://numericalmethods.eng.usf.edu/rights.htm

Table of Contents

Method

Nonlinear models using least squares

Exponential model

Polynomial model

Linearized data model

Power functions

Growth model

Example

Example 1: Radioactive material decay

Example 2: Height of child vs age

Example 3: Thermal expansion vs temperature

Example 4: Radioactive material decay with data linearization

Example 5: Overpotential vs current

Example 6: Chemical rate reaction vs. concentration

Presentation

Power Point Presentation

Simulation

Without Data Linearization

[MAPLE]  [MATHCAD]  [MATHEMATICA]  [MATLAB]

With Data Linearization

[MAPLE]  [MATHCAD]  [MATHEMATICA]  [MATLAB]

Polynomial Regression

[MAPLE]  [MATHCAD]  [MATHEMATICA]  [MATLAB]

Comparing with and without Data Linearization

[MAPLE]  [MATHCAD]  [MATHEMATICA]  [MATLAB]

From fundamental theories, we may know the relationship between two variables. An example in chemical engineering is the Clausius‑Clapeyron equation that relates vapor pressure, P of a vapor to its absolute temperature, T.

(1)

where A and B are the unknown parameters to be determined. The above equation is not linear in the unknown parameters. Any model that is not linear in the unknown parameters is described as a nonlinear regression model.

Back to TOC

Nonlinear models using least squares

The development of least squares estimation for nonlinear models does not generally yield equations that are linear and hence easy to solve. An example of a nonlinear regression model is the exponential model.

Exponential model

Given , , . . . , best fit to the data. The variables and are the constants of the exponential model. The residual at each data point is

(2)

The sum of the square of the residuals is

(3)

To find the constants a and b of the exponential model, we minimize S_r by differentiating with respect to and and equating the resulting equations to zero.

(4a,b)

or

(5a,b)

Equations (5a) and (5b) are nonlinear in a and b and thus not in a closed form to be solved as was the case for the linear regression. In general, iterative methods (such as Gauss‑Newton iteration method, Method of Steepest Descent, Marquardt's Method, Direct search, etc) must be used to find values of a and b.

However, in this case, from Equation (5a), can be written explicitly in terms of as

(6)

Substituting Equation (6) in (5b) gives

(7)

This equation is still a nonlinear equation in and can be solved best by numerical methods such as bisection method or secant method.

Worksheet for regression without data linearization [MAPLE]  [MATHCAD]  [MATHEMATICA]  [MATLAB]

Back to TOC

Example 1

Many patients get concerned when a test involves injection of a radioactive material. For example for scanning a gallbladder, a few drops of Technetium-99m isotope is used. Half of the techritium-99m would be gone in about 6 hours. It, however, takes about 24 hours for the radiation levels to reach what we are exposed to in day-to-day activities. Below is given the relative intensity of radiation as a function of time.

Table 1 Relative intensity of radiation as a function of time

t(hrs)	0	1	3	5	7	9
	1.000	0.892	0.708	0.562	0.447	0.355

If the level relative intensity of radiation is related to time via an exponential formula , find

a) The value of the regression constants and .

b) The half-life of Technium-99m?

c) Radiation intensity after 24 hours?

Solution

a) The value of is given by solving the nonlinear Equation (7),

(8)

and then the value of A from Equation (6),

(9)

Equation (8) can be solved for using bisection method. To estimate the initial guesses, we assume and . We need to check whether these values first bracket the root of . At , the table below shows the evaluation of .

Table 2 Summation value for calculation of constants of model

1 2 3 4 5 6	0 1 3 5 7 9	1 0.891 0.708 0.562 0.447 0.355	0.00000 0.70719 1.0620 0.88509 0.62087 0.39937	1.00000 0.70719 0.35400 0.17702 0.08870 0.04437	1.00000 0.62996 0.25000 0.09921 0.03937 0.01562	0.00000 0.62996 0.75000 0.49606 0.27560 0.14062
			3.6745	2.3713	2.0342	2.2922

From Table 2

Similarly

Since

,

the value of falls in the bracket of . The next guess of the root then is

Continuing with the bisection method, the root of is found as . This value of the root was obtained after 20 iterations with an absolute relative approximate percentage error of less than 0.0002%.

From Equation (9), A can be calculated as

= 0.99983

The regression formula is hence given by

b) Half life of Technetium-99m is when

c) The relative intensity of the radiation after 24 hrs is

This implies that only of the initial radioactive intensity is left after 24 hrs.