Time Series Forecasting: Moving Beyond Classical Models in the Univariate Case

Mastering Time Series Components & Models

Author: Andrew McQueen

23 September, 2024

In this blog, our goal is to explain some of the techniques that go beyond classical time series forecasting models. Importantly, we will explain regression as basis for prediction on temporal data and break down the components of time series. 

For information on the different components of time series and models from naïve to ARIMA, see our other blogs: 

• Time Series Forecasting, An Introduction 
• Time Series Forecasting Using Classical Models 

Feature Engineering 

In our last blog, we explained classical models in the univariate case without exogenous variables. Those models can often be used by having a time variable with a consistent period, or interval, and the variable we are interested in forecasting at that interval. For time series forecasting, feature engineering includes the processes of data preparation and variable creation. For example, when training a multiple linear regression model for time series, we might include month and year variables to model the effects of the two and to understand whether yearly seasonal effects occur. We also might use transformations to improve a model’s ability to fit the data in the case of an exponential relationship between time and our dependent variable. If we suspect autocorrelation, we add lagged variables to model the autoregressive component. 

Machine Learning 

Machine learning offers many techniques for using data to solve problems. With quantitative temporal data, regression will be our solution. Regression is used for prediction and inference when relationships between two or more variables exist, and the output is quantitative. In the time series context, we can see this as the relationship between time and some other variable, like sales. 

Some regression techniques are great for interpretability, while others are more focused on predictive ability. A simple model with reasonable assumptions invites us to interpret model coefficients and variable significance to explain the relationships between independent variables and the dependent variable. Interpretability is particularly useful when stakeholders require an. More complex models typically offer less in terms of interpretability but a deeper understanding of patterns and relationships in the data and can offer better predictions if parameters are effectively optimized.  

Revisiting the Components 

As mentioned in our introductory blog, time series are made up of different components. This allows us to split up and build models for each component. Using decomposition techniques, our goal is to isolate the behaviors of these components. When using classical models, we can think of Holt-Winters’ Exponential Smoothing method as a single model in which the parameters are the level, trend, and seasonality. And this can be broken up into—for example—a sinusoidal pattern throughout each week, increases in warm months and decreases in cold months, and an exponential increase over the years. Smaller intervals within the historical data can contain useful information about the specific patterns and characteristics of each component. 

For instance, looking at weekly or daily data may reveal cyclical patterns related to the day of the week or other weekly rhythms. Analyzing monthly or quarterly data could uncover seasonal trends tied to changes in the calendar year, such as sales spikes around the holidays. And examining the long-term trend over years or decades may show more gradual shifts like exponential growth or decline. 

Carefully analyzing the different time horizons and isolating the unique behaviors of each component is key to building accurate and insightful time series models. The information contained within these smaller time intervals can provide valuable clues about the underlying drivers and dynamics of the overall time series. 

Exogenous Variables 

The time series in which we are interested is called the endogenous time series. When adding information not pertaining to it, we are adding exogenous time series. Exogenous (think external) variables require some relationship to the endogenous variable to add value to the forecast. Knowing when and what to include in these other sources of information typically requires domain knowledge. For example, electricity use and clothing sales will likely depend on very different external variables.

Time series models, like SARIMAX, offer the ability to add information to our models through exogenous variables. Importantly, we need to make assumptions about these variables. Since adding this information creates relationships with the time series, we either need to forecast the exogenous variables or include assumptions about their behavior over the forecast horizon.  

Ensembles 

No one single forecasting model consistently outperforms others across all scenarios. Ensembles give us a chance to combine predictions from multiple models, improving the accuracy and robustness of the forecast through diversification. Ensembles can be helpful in situations where many time series need to be modeled quickly. They reduce the potential variance of one single model, as an ETS model may perform well on some series but poorly on others.  Some options for ensembles include: 

• Weighted Average Ensembles 
• Stacking 
• Hybrid Models 

For weighted average ensembles, choosing the weights can be a simple average or a weight based on historical individual model performance. Stacking techniques involve using the individual models as features of a regression model, where the weights would be handled by the chosen method of regression. Hybrid models combine classical and machine learning models for understanding different components of the time series. An example of a hybrid model is when ETS or ARIMA could model the linear trend of the time series, while a non-linear regression model could capture the remaining non-linear relationships. 

Ensemble methods are a good choice—particularly in complex forecasting scenarios or when building models for many time series at once—because they leverage the strengths of multiple models and mitigate individual model weaknesses through diversification. 

Conclusion 

Moving beyond classical models opens a world of possibilities for time series forecasting. Feature engineering, machine learning, and ensemble techniques provide tools to capture more intricate patterns in temporal data, leading to more accurate and reliable forecasts. However, these methods also require a deeper understanding of the data and more computational resources. 

As we’ve explored, modern time series forecasting is not just about choosing the right model but also about understanding the data’s underlying structure, selecting relevant features, and combining multiple approaches to achieve the best results. They also provide a method for handling forecasting many time series for when time does not allow for manual decomposition and parameter estimation.