Regression Assumptions

The basic regression model (as well as more complicated ones) have certain underlying assumptions, violations of which have an impact on the optimality of the fitted model, i.e., the extent to which the model and its parameters represent the best model that can be fit, that is, the one that performs the best in the tasks of representing the relationship between the response variable and the predictor variable(s), or predicting future values of the response variable given new values of the predictor variables.

• regression assumptions, and the consequences of their violations

Multiple Regression

Multiple regression is (conceptually) a simple extension of bivariate regression, in which the influence of more than one predictor variable on the response can be estimated.   For the case with two predictor variables, the analysis can be thought of as involving the fitting of a plane (as opposed to a line in the bivariate regression case), and the equations for the OLS estimates of the regression equations are only a little more complicated algebraically.  For three or more predictors, the algebra is also quite simple, but requires the use of matrix algebra.

Fitting a multiple regression equation

A couple of illustrations jointly describe the idea of fitting a plane

• fitting a plane using two predictor variables
• one data point and its deviation from the regression plane or response surface

The mathematics behind multiple regression analysis is more complicated than that for bivariate regression, but can be elegantly presented using matrix algebra.

• regression in matrix-algebra format

Example

• an example of the application of multiple regression, and diagnostic testing of the fitted models

Some More Examples:

• Dummy-variable regression

Readings: 

Kuhnert & Venebles (An Introduction...):  p. 109-120;  Maindonald (Using R...):  ch. 5