Smoothing and nonparametric regression

A review of global fitting (e.g. linear regression)

In ordinary linear regression analysis, the objective can be considered to be drawing a line through the data in an optimal way, where the parameters (regression coefficients) are determined using all of the data, i.e. they are globally determined.  However, it is possible to think of the line as connecting the points, that for each value of X, represent the local density maxima of Y--it just happens that these local maxima happen to be arranged along a straight line.

• the scatter diagram and fitted line
• the conceptual model that underlies the interpretation of the line
• curvilinear regression

Loess curves

A bivariate smoother is a function or procedure for drawing a smooth curve through a scatter diagram.  Like linear regression (in which the "curve" is a straight line), the smooth curve is drawn in such a way as to have some desirable properties.  In general, the properties are that the curve indeed be smooth, and that locally, the curve minimize the variance of the residuals or prediction error.

The bivariate smoother used most frequently in practice is known as a "lowess" or "loess" curve.  The acronyms are meant to represent the notion of locally weighted regression--a curve- or function-fitting technique that provides a generally smooth curve, the value of which at a particular location along the x-axis is determined only by the points in that vicinity.  The method consequently makes no assumptions about the form of the relationship, and allows the form to be discovered using the data itself.  (The difference between the two acronyms or names is mostly superficial, but there is an actual difference in R--there are two different functions, lowess() and loess(), which will be explained below.)

The mechanics of loess:

• the locally weighted "fitted value" at a particular point
• what happens at the margin of the scatter diagram
• the mathematics of loess
• the parameters of an individual curve (in addition to smoothing type)
• • span -- a value between 0 and 1 controlling the amount of smoothing, with smaller values resulting in less smoothing.  Typical values lie in the range of .3 to .5.  Span can be considered to represent the width of the smoothing window.
  • degree -- the degree of the locally-fitted polynomial.  1 = locally linear fitting (i.e. a line), 2 = locally quadratic quadratic fitting (i.e. a parabola).
  • family -- either "Symmetric" or "Gaussian"  The symmetric option combines the local fitting with a "robustness" step that discounts the influence of unusual points, while the Gaussian option does not.