Regression assumptions

There are a few assumptions that underlie regression analysis:

• the prediction errors or residuals are assumed to be independent, identically normally distributed random variables, with a mean of 0 and a standard deviation of s,
• the X's (predictor or independent variables) are known without error,
• the X's are not correlated.
• the correct model has been specified (i.e. the right predictors have been included in the model.

If the assumptions are not violated, then the Gauss-Markov theorem indicates that the OLS estimates (e.g. eqns 13 and 14) are optimal in the sense of being unbiased and having minimum variance. If one or more of the assumptions are violated, then estimated regression coefficients may be biased (i.e. they may be systematically in error), and not minimum variance (i.e. there may be more uncertainty in the coefficients than is apparent from the results).

Consequences of assumption violations

If the assumptions are violated, then there may be two consequences--the estimated coefficients may be biased (i.e. systematically wrong), and they may longer have minimum variance (i.e. their uncertainty increases).

• the notion of variability of the regression coefficients
• illustrations using repeated simulations of data sets with built-in assumption violations. (violate1.gif, violate2.gif)
  
  

Testing for assumption violations with diagnostic analyses

There are several ways to test whether the assumptions that underlie regression analysis have been violated.  As might be expected, these include analytical methods, in which the values of particular test statistics are computed and compared with appropriate reference distributions, and graphical methods, which are often easier to implement (and just as informative).

• residual plots and case-wise statistics

• some examples with another example data set [regrex3.csv]