A Comparison Study of Ridge Regression and Principle Component Regression with Application

Abstract

The purpose of this paper is to discuss the multicollinearity problem in regression models and presents some typical ways of handling the collinearity problem. In Addition, the paper attempts to compare RR , and PCR and LS methods using minimum squared error MSE and the accuracy of the prediction. The results of this paper showed that, RR  method performs better than PCR and LS methods , because RR had minimum MSE and a higher predicted accuracy than other methods. The results of this paper showed that, based on the criteria of model accuracy PCR performs better than RR, whereas, according to mean squares errors criterion MSE , RR performs slightly better. In general, the two biased estimator RR and PCR perform better than LS.

Keywords: Least Squares; Correlation Matrix; Multicollinearity; Ridge Regression; Principal Component Regression.