M4-Competition for time series forecasting, daily data.
From original source:
-----
The fourth competition, M4, started on 1 January 2018 and ended in 31 May 2018.
The M4 extended and replicated the results of the previous three competitions, using an extended and diverse set of time series to identify the most accurate forecasting method(s) for different types of predictions. It aimed to provide answers on how to improve forecasting accuracy and identify the most appropriate methods for each case. To get precise and compelling answers, the M4 Competition utilized 100,000 real-life series, and incorporated all major forecasting methods, including those based on Artificial Intelligence (Machine Learning, ML), as well as traditional statistical ones.
-----
The time series were downloaded via the R package M4comp2018 and then loaded with a python script to obtain the different datasets:
'Yearly', 'Quarterly', 'Monthly', 'Weekly', 'Daily', 'Hourly'. The data in R already gives us some kind of date as the index for the time series.
The required number of forecast values to be forecasted for each time series, for the 'Daily' dataset was always 14. Therefore, if one wants to evaluate
their model to be compared with other models from the original competition, the last 14 values of each time series are considered the test dataset.
Note that the participants did not have access to the date of the time series during the competition. Besides, some dates are ambiguous due to
the representation of only 2 digits for the year (XX-XX-17 could represent 1817, 1917, 2017 etc).
There are 5 columns:
id_series: The id of the time series.
date: The date of the time series in the format "%Y-%m-%d".
time_step: The time step on the time series.
covariate_0: Covariate values of the time series, tied to the 'id_series'. Not interested in forecasting, but can help with the forecasting task.
value_0: The values of the time series, which will be used for the forecasting task.
Preprocessing:
1 - The original index were the number of days since the unix epoch (01-01-1970), we transformed it in the 'date'.
2 - Renamed 'Category' to 'covariate_0'.
3 - Created column 'time_step' with increasing values of time step for the time series.
4 - Casted 'date' to str, 'time_step' to int, 'value_0' to float, and defined 'id_series' and 'covariate_0' as 'category'.