FRED-MD

Monthly Database for Macroeconomic Research From original website: ----- FRED-MD and FRED-QD are large macroeconomic databases designed for the empirical analysis of 'big data'. The datasets of monthly and quarterly observations mimic the coverage of datasets already used in the literature, but they add three appealing features. They are updated in real-time through the FRED database. They are publicly accessible, facilitating the replication of empirical work. And they relieve the researcher of the task of incorporating data changes and revisions (a task accomplished by the data desk at the Federal Reserve Bank of St. Louis). The accompanying papers shows that factors extracted from the FRED-MD and FRED-QD datasets share comparable information content to various vintages of so-called Stock-Watson datasets. These factor estimates are shown to be useful for forecasting a wide range of macroeconomic series. In addition, we find that diffusion indexes constructed as the partial sum of the factor estimates can potentially be useful for the study of business cycle chronology. ----- We used the file 2024-05.csv for monthly data and performed some preprocessing. There are 119 columns: id_series: The id of the time series. date: The date of the time series in the format "%Y-%m-%d". time_step: The time step on the time series. value_X (X from 0 to 115): The values of the time series, which will be used for the forecasting task. Preprocessing: 1 - We transformed each column following the "Tranform" code available in the first line of the file and specified in the original paper which are: 1 - no transformation; 2 - Delta(xt); 3 - Delta^2(xt); 4 - log(xt); 5 - Delta(log(xt)); 6 - Delta^2(log(xt)); 7 - Delta(xt/xt-1 - 1.0) 2 - Dropped first 3 rows to get rid of NaNs due to the transformations and last 4 rows to only consider dates until 2023 (there are some missing values for 2024). 3 - Standardize the 'sasdate' column to the format %Y-%m-%d and renamed it to 'date'. 4 - Dropped columns that only have values starting later than 1959-03-01 (8 of 127 columns). 5 - Created column 'id_series', with value 0, there is only one multivariate series. 6 - Ensured that there are no missing dates and that they are evenly spaced (monthly). 7 - Created column 'time_step' with increasing values of the time_step for the time series. 8 - Filled NaN values with the last valid observation (only 2 cases for columns COMPAPFFx and CP3Mx in 2020-04-01 and 2020-05-01). 9 - Renamed columns to 'value_X' with X from 0 to 115. 10 - Casted 'date' to str, 'time_step' to int, 'value_X' to float, and defined 'id_series' as 'category'.