Chapter 5: Model Building and Diagnostics in Regression

Abstract:

As mentioned in Section 2.3, the regression of y on x₁, …, x_k, namely E(y❘x₁, …, x_k) represents the ‘best’ approximation (in the sense of smallest mean squared error) of a random variable y, as a function of other random variables x₁, …, x_k. The particular linear model (y, X β, σ² I) presents us with a simple setup for studying such a regression function and the associated approximation error. Specifically, the model for a single response variable y becomes

and the observations on y are regarded as uncorrelated random variables (see (1.7)). While Chapter 3 discussed how the parameters β and σ² can be estimated, Chapter 4 considered various inferential issues such as testing of hypothesis, construction of confidence intervals and regions, prediction, construction of tolerance and prediction intervals etc., under the additional assumptions that the conditional distribution of y given x₁, …, x_k is normal and that these observations are independent. Such a streamlining of these inference procedures makes the linear model a very popular choice in studying regression problems…