OLS estimators

This article is part of the EconHelp Tutoring Wiki

In statistics and econometrics, ordinary least squares (OLS) or linear least squares is a method for estimating the unknown parameters in a linear regression model. This method minimizes the sum of squared vertical distances between the observed responses in the dataset, and the responses predicted by the linear approximation. The resulting estimator can be expressed by a simple formula, especially in the case of a single regressor on the right-hand side.

The OLS estimator is consistent when the regressors are exogenous and there is no multicollinearity, and optimal in the class of linear unbiased estimators when the errors are homoskedastic and serially uncorrelated. OLS can be derived as a maximum likelihood estimator under the assumption that the errors are normally distributed, however the method has good statistical properties for a much broader class of distributions (except for efficiency).

Properties of OLS estimators

OLS estimators have the following properties

• Unbiasedness
• Minimum Variance
• Efficiency

Unbiasedness

Suppose that the population size is 100 for anything that we are studying. We use samples of size 10 to estimate the α and β of the population. Everytime we use a different sample (a different set of 10 unique parts of the population), we will get a different α and β. With the OLS method of getting α and β, we get a situation wherein after repeated attempts of trying out different samples of the same size, the mean (average) of all the α and β from the samples will be equal to the actual α and β of the population as a whole. Basically, this means that if you do the exercise over and over again with different parts of the population, and then you find the mean for all the answers you get, you will have the correct answer (or you will be very close to it).

Minimum variance

This property is what makes the OLS method of estimating α and β the best of all other methods. In the previous section you learned about the principle of unbiasedness. It is possible, however, that two different methods of estimation could be unbiased. How do we decide which one to use in our econometric model? This is where the principle of variance is involved. Variance is a measure of how far the different α and β are from their mean. An estimator (a function that we use to get estimates), that has a lower variance is one whose individual data points are those that are closer to the mean. This estimator is statistically more likely than others to provide accurate answers. The OLS estimator is one that has a minimum variance.

Efficiency

This property is simply a way to determine which estimator to use.

• An estimator that is unbiased but does not have the minimum variance is not good.
• An estimator that has the minimum variance but is biased is not good
• An estimator that is unbiased and has the minimum variance of all other estimators is the best (efficient).

The OLS estimator is an efficient estimator.