Articles

Mastering Time: How to tackle your time series forecasting problems without losing your mind

9 min read

Time series problems are extremely common, for example, sales forecasting, energy demand prediction, asset valuation, weather prediction, etc. The temporal factor makes them unique, with their own challenges and solutions.

Interest in time series has grown over the years, but not exponentially, unlike areas such as Machine Learning or Deep Learning. This raises the question: Why hasn’t interest in time series grown in the same way ML or DL has?

One possible answer to the previous question is that the results produced by time series are less “impactful” than those obtained with other models. But without a doubt, the main explanation is that time series have particular characteristics that make their workflow more tedious. Below, we will go through each of these particularities that make working with time series more complex, and how to address them.

Training with few examples

Time series are trained with few examples, only a few series for each model. The problem is that with few validation data, it is easy for generalization and overfitting issues to occur in the models.

Solutions include using cross-validation with rolling windows or a sliding window approach. Cross-validation can help build better models because training periods vary in length, while sliding windows keep training sets fixed. Simpler models such as linear regression can also be used to avoid overfitting.

If cross-validation is used, the advantage is that the model will likely perform better because the training periods vary in length, the model trained on the longest period will generally be stronger. With the sliding window, we leave behind data that would not be excluded in the real model, although the best solution will always depend on the type of data available.

There is also the option of using simpler models (models that do not overfit), such as linear regression, which is rarely used nowadays in other Machine Learning problems.

Of course, it is important to manage expectations. We must be aware that with limited data, it is not possible to make a highly accurate prediction.

Sparsity

Many times in time series we encounter periods without data, for example, if we talk about orders to a warehouse, there will be days when there are no orders and therefore no data.

To solve this problem, you can use a zero-inflated regression, a model based on assuming that non-zero data follow a certain distribution, train a model using only the non-zero values and then add the zeros, use a Tweedie loss function (a loss function based on a probability distribution with many zeros), or discuss with the client whether a top-down approach or temporal aggregation would be appropriate (for example, forecasting weekly instead of daily).

Multiple series with different behaviors

Sometimes you have to predict many time series and they can be very different in terms of dynamics, predictability, the problem to solve, etc.

In these cases, different techniques, models, and parameters are usually applied to each time series. Another option is to use a clustering model to identify which series are most similar to one another, allowing the same model to be used for those groups and providing more training data than if a single model were used per series. Finally, you can also separate the time series with the best performance from those with poor predictions: train the good ones together to avoid being influenced by the bad series, and train the poorly performing time series jointly using all the available data.

Butterfly Effect

In time series forecasting, when predicting one step from the previous one, the error from the earlier prediction can carry over to the next due to feedback. As a result, the error accumulates and propagates forward, growing uncontrollably. This effect is more likely to occur the more unpredictable the time series is.

To solve this problem, you can create models that predict the value at different horizons. In other words, build separate models for each forecasting horizon to avoid feeding error back into the next step. The downside of this approach is that it requires creating and training more models. Another option is to use a single model whose output is a vector covering all the horizons you want to predict. In this case, the model would be more complex, but it would solve the problem of error propagation.

Missing Values

It is very common to encounter missing values in time series, but unlike other problems, time series containing these values usually cannot be discarded. If we removed this information, we would be left with no data. That is why missing values are typically imputed. Filling isolated missing points is straightforward, but the task becomes more complex when there are long stretches of missing data or when these gaps appear at the beginning or end of the series.

If we have isolated missing values, one option is to use a sliding window and fill them with the mean or median. If instead we need to fill long stretches of missing data, we could use linear interpolation (linearly connecting the points before and after the missing interval) or Splines. This technique has the advantage of providing continuity (in the case of Splines, it preserves continuity in the derivative of the series), but it may completely change the dynamics of the series. Therefore, what is usually done in these cases is to use a dedicated time series forecasting model to fill in the missing values. This ensures that the trend and seasonality of the imputed values match those of the rest of the series.

Outliers

Again, when we encounter anomalous values in a time series, we cannot simply discard these data points. Once the outliers are detected, if they are identified as errors, they should be treated as missing values and handled in the way described in the previous section. However, it is important to highlight that in time series, erroneous values cannot be detected based on their absolute magnitude (i.e., being very high or low compared to the rest of the series). Instead, they must be detected within a temporal context (meaning the value must be significantly higher or lower than what would be expected based on the series’ patterns and trends).

One possible solution to this problem is to decompose the series into its three components: seasonality, trend, and residual, analyze the residual, and detect as an outlier any residual value that exceeds a certain threshold. Another very common solution is to use the Hampel filter, where for each point the median of its moving window and the standard deviation are calculated. In this case, the outlier will be the point whose value differs from the median by more than X standard deviations. Once the outliers are detected, they can be imputed using the median. Finally, other solutions include using Generalized Extreme Studentized Deviate (ESD), Exponentially Weighted Moving Average, or Isolation Forest to detect outliers.

Input Features

Another problem we might have is deciding which inputs to use in our time series model.

You can include relevant external or internal information as input (previous history, changes in available variables, information already present in the time series, etc.). For example, when forecasting beach umbrella sales, useful input data could include: sales from the same period the previous year, the probability of sunny weather, or whether umbrella sales tend to increase or decrease on weekends (a pattern that could be observed directly in the time series).

Validation and test periods

In time series problems, a validation period and a test period are used. The validation period is used during training to tune the model’s hyperparameters and find the best model. It is usually split into training and validation sets.

Once the hyperparameters that best fit the model have been selected, the model is retrained using the training + validation set. Finally, the test period is used to evaluate how the model performs on unseen data.

Due to the temporality of the data, the validation and test periods cannot be the same. The test period comes after the validation period in time.

Special Days (Holidays or Events)

These events can significantly affect the time series and therefore the prediction error. These dates usually do not follow a completely regular pattern. Some follow a regular pattern, such as annual holidays (for example, October 12 is a holiday every year), but depending on the day of the week it falls on, it may create a long weekend, changing the data’s behavioral dynamics. It is also important to consider that national and local holidays can have different impacts. Additionally, marketing campaigns or promotions should be taken into account, as they can influence the data when, for example, forecasting product sales.

To address this problem, it is necessary to review the historical data and determine whether special days have affected the time series being forecasted. It is important to know that some libraries include built-in calendars, which can be very useful. You can also use features from similar days as inputs (a day that has high correlation and similar behavior to the day you want to predict). Another option is to use different models to forecast normal days and special days.

At this point, of course, it is important to be realistic and understand when an event like this is predictable or not, for example, spontaneous events.

The most common groups of models used for time series problems are: Linear Regression models (very fast and easy to train), ARIMA (which accounts for seasonality and trend), Prophet (which also handles seasonality and trend, includes a holiday calendar, and provides strong handling of missing or erroneous data), Recurrent Neural Networks such as GRU or LSTM (they maintain an internal state containing historical information, enabling better predictions when history is important), Boosting models like XGBoost or CatBoost (capable of capturing complex relationships between inputs and the target), N-BEATS/N-HITS (neural networks with some interpretability), and TFT models (Transformer-based, interpretable, multi-horizon, and offering many additional advantages). It is important to be familiar with them in order to understand which model is best suited depending on the input data available, the time available to produce the forecasts, and so on.

It is clear that the time factor adds an extra layer of difficulty to problems involving time series. That is why in this type of models it is necessary Carry out a good analysis and preprocessing of the data. The quality of the predictions will depend much more on these preliminary stages. depending on the model that is ultimately used. The more knowledge you have about the data (which inputs to use, which events affect the series or not, etc.), the better the predictions will be.

You can watch the talk “Mastering Time: How to Tackle Your Time Series Forecasting Problems Without Losing Your Mind” presented at PyCon ES 2023 here:

https://www.youtube.com/watch?v=edcCT0U9u7g
CONTACT US

Our experts can help you

Tell us the constraint you are working against: the regulator, the core system, the data, and we will tell you where we would start.