Least Squares Method
Suppose we have a set of data, that is a set of points
, which
come from an experiment. See below
If we wish to compute the ``error'' in fitting a straight line, we could
compute the shortest distance to a line (see below)
It is, however, more convenient to let the error be measured by the vertical deviation from the straight line, as shown below.
The vertical distance
from a point
to the line y=mx+b is given by
And the total distance is
In order to obtain the ``best'' line, we want to make this error as small as possible. Minimizing the distance is the same as minimizing the distance squared, we define the error to be minimized as
Clearly, the error is a function of two variables, the slope (m)
and the intercept (b). Since the error is a quadratic function
of two variables, and is always positive,
it must have a unique minimum! The minimum is achieved at the
point
and
which satisfy
The quantities
,
,
and
are crucial
to calculate. In terms of A, B, C, D, we have
We can solve these equations by substitution or by cross multiplication and subtraction:
therefore,
and
Dividing numerator and denominator by
we have
Solving for
from the first equation, we get
These general formulas are valid for an arbitrary amount of data, and
can easily be programmed in maple or any other computer language.