Learning nonlinear features to improve linear forecasting approaches

Abstract

Forecasting of future events is critical for decision making in many industries. Especially in the retail industry, forecasting of future sales has critical importance for management of the company. General time series models are a widely used method for forecasting. However, since general time series models mostly consider linear re lation between response and explanatory variables, they can miss nonlinear relations which can have a critical effect on the response variable. We propose an iterative ap proach that starts with a base model and explain the residuals by tree-based regression. The path leading to the highest error is added to the base model as a new variable. Proposed algorithm is an improvement on general time series models since it adds nonlinear variables by residuals explanation to the linear models in the second stage. Proposed model consists of two-stage; first stage is a general time series model where Autoregressive Integrated Moving Average with regressor version (ARIMAX), Linear Regression and Penalized Regression models were used as base learner in this study, second stage is a residual explanation by regression tree to find new explanatory vari ables that cause the highest error on the first stage by considering linear and nonlinear relations. New regressors which are found on the second stage are added to the first model and new model continues for forecasting until it optimizes itself. Implementa tion of proposed algorithm on ARIMAX outperformed on regular ARIMA model and ARIMAX model with same regressors on the proposed model. Also, proposed algo rithm was implemented on Linear Regression and Penalized Regression methods and when compared with regular Linear Regression and Penalized Regression respectively, proposed algorithm achieved better results.