[Deep Learning] Time Series Analysis Tool TSLiB Library Usage Guidelines

TSLiB is an open source repository tailored for deep learning time series analysis. It provides a unified implementation of a variety of deep time series models, making it easy for researchers to evaluate existing models or develop customized models.TSLiB covers five mainstream time series tasks, including longterm forecasting, short-term forecasting, missing value TSLiB covers five mainstream time series tasks, including Long-term forecasting, Short-term forecasting, Missing value filling, Anomaly detection and Classification, and is an ideal tool for researchers engaged in time series analysis.The official address of the TSLiB library can be found at:Time-Series-Library。

1 Use of the TSLiB library

1.1 Introduction to Timing Tasks

The five timing tasks supported by TSLiB are summarized below:

Type of mission	define	specificities	Examples of application scenarios
long-range forecast	Forecasting the trend of the time series over a longer period of time in the future	Long-term trends and seasonality need to be taken into account, using sophisticated models to capture long-term dependencies	Stock price forecasts, long-term energy demand forecasts, etc.
short-term forecast	Forecasting future values of a time series in the near future	Often focuses on short-term fluctuations and models need to respond quickly to new data	Short-term sales forecasts, traffic flow forecasts, etc.
Missing value filling	Filling in missing data points in a time series	Need for continuity and consistency of time series	Time series preprocessing, historical data completion, etc.
anomaly detection	Identify anomalies or outliers in a time series	Need to distinguish between normal fluctuations and unusual events	Network security monitoring, equipment failure detection, etc.
categorization	Classify time series data into different categories or labels	Classification is usually based on the characteristics of the time series	Customer behavior analysis, medical diagnosis, etc.

At the time of writing, TSLiB pooled the top three performing models as of March 2024 on five different tasks as follows:

mandates	firstly	secondly	tertiary
Long-Term Forecast - Past 96 Steps	iTransformer	TimeMixer	TimesNet
Long-Term Forecast-Search	TimeMixer	PatchTST	DLinear
short-term forecast	TimesNet	Non-stationaryTransformer	FEDformer
Missing value filling	TimesNet	Non-stationaryTransformer	Autoformer
categorization	TimesNet	FEDformer	Informer
anomaly detection	TimesNet	FEDformer	Autoformer

The deep learning timing models in the TSLiB library that participated in the above rankings are as follows:

• TimeMixer: TimeMixer is a novel architecture for time series forecasting that captures complex patterns in time series data by decomposing the multi-scale mixing operation[ICLR 2024] [Code]。
• TSMixer: TSMixer is a fully connected multilayer perceptron (MLP) architecture for time series forecasting. It efficiently extracts information through a mixture of operations along time and feature dimensions[arXiv 2023] [Code]。
• iTransformer: iTransformer presents an inverted Transformer architecture for time series forecasting. It captures multivariate correlations by applying attention and feedforward networks in the inverted dimension and learns nonlinear representations[ICLR 2024] [Code]。
• PatchTST: PatchTST is a Transformer-based long-time forecasting model for time series, which treats the time series as 64 tokens consisting of multiple variables[ICLR 2023] [Code]。
• TimesNet: TimesNet is a model for generalized time series analysis that performs forecasting by modeling two-dimensional changes in time series[ICLR 2023] [Code]。
• DLinear: DLinear is a linear time series forecasting model that explores the effectiveness of Transformer in time series forecasting[AAAI 2023] [Code]。
• LightTS: LightTS is a fast multivariate time series forecasting model that employs a lightweight sample-oriented MLP structure[arXiv 2022] [Code]。
• ETSformer: ETSformer is a time series forecasting model that combines exponential smoothing and Transformer[arXiv 2022] [Code]。
• Non-stationary Transformer: Non-stationary Transformer is a model to explore non-stationarity in time series forecasting[NeurIPS 2022] [Code]。
• FEDformer: FEDformer is a frequency-enhanced decomposition Transformer model for long-time series prediction[ICML 2022] [Code]。
• Pyraformer: Pyraformer is a low-complexity pyramidal attention model for long time series modeling and forecasting[ICLR 2022] [Code]。
• Autoformer: Autoformer is a Transformer model with autocorrelation decomposition for long time series prediction[NeurIPS 2021] [Code]。
• Informer: Informer is a classical model for long time series time series forecasting[AAAI 2021] [Code]。
• Reformer: Reformer is an efficient Transformer model that reduces computational complexity by using locally sensitive hashes[ICLR 2020] [Code]。
• Transformer: Transformer is a model based on the self-attention mechanism, which has achieved great success in several fields such as natural language processing[NeurIPS 2017] [Code]。

There are also a number of deep learning timing models that are included in the TSLiB library, but have not yet participated in the TSLiB library performance rankings. You can follow the official TSLiB link for the latest results:

• Mamba: Mamba is a linear time series model based on a selective state space. the Mamba model excels when dealing with long series data and achieves state-of-the-art performance on a wide range of modes[arXiv 2023] [Code]。
• SegRNN: i.e., Segment Recurrent Neural Network, a recurrent neural network for long time prediction[arXiv 2023] [Code]。
• Koopa: Koopa model for learning non-stationary time series dynamics using Koopman predictor[NeurIPS 2023] [Code]。
• FreTS: A model for applying multilayer perceptrons (MLPs) in the frequency domain for time series forecasting[NeurIPS 2023] [Code]。
• TiDE: i.e., Time-series Dense Encoder, a multilayer perceptron (MLP) base codec model for long time-series prediction[arXiv 2023] [Code]。
• FiLM: i.e., Frequency improved Legendre Memory Model, which combines the Fourier transform and Legendre polynomials to improve the model's ability to capture the frequency characteristics of time series.[NeurIPS 2022][Code]。
• MICN: i.e., Multi-scale Local and Global Context Modeling, which captures the complex dynamics of time series through multi-scale local and global context modeling[ICLR 2023][Code]。
• Crossformer: Crossformer is a Transformer model that exploits cross-dimensional dependencies for multivariate time series forecasting, which captures complex relationships between different time series through a cross-dimensional self-attention mechanism[ICLR 2023][Code]。
• TFT: i.e., Temporal Fusion Transformers, a model for interpretable multi-objective time series forecasting[arXiv 2019][Code]。

1.2 Introduction to the TSLiB library

1.2.1 Environment installation

The structure of the entire project is illustrated in the figure below:

The installation code is as follows:

git clone /thuml/Time-Series-Library
pip install -r 

This paper introduces the use of the TSLiB library based on the TimesNet model.The TimesNet model supports long-time prediction, short-time prediction, missing-value filling, anomaly detection, and classification, and is also a model proposed by the TSLiB author team.TimesNet is described as follows:TimesNet: a time-series base model with five leading tasks such as prediction, filling, and classificationFor a tutorial on the use of the TSLiB library based on the TimesNet network see:TimesNet_tutorial.ipynb。

1.2.2 TimesNet network utilization

TimesNet is a general-purpose time series neural network that handles a variety of different time series tasks. As can be seen from the associated library loading of TimesNet, its key components include the fft function, which implements the Fast Fourier Transform, and the Inception feature extraction backbone network.

import torch    
import  as nn
import  as F
import 
from  import DataEmbedding
from layers.Conv_Blocks import Inception_Block_V1   

Fast Fourier Transform-based Data Transformation

The core concept of TimesNet is to realize the transformation from one-dimensional data to two-dimensional data. Nowadays, most existing methods mainly focus on the temporal dimension of time series to obtain the temporal dependence. The original one-dimensional series, which is similar to a straight line with points arranged sequentially, mainly presents the direct changes between neighboring time points. For example, yesterday's data is 0 and today's data is 12, only a simple increase or decrease can be perceived. However, in fact, real-world time series usually have multiple cycles, such as daily, weekly, monthly, etc.; and there are dependencies between time points within each cycle (e.g., today's 10 o'clock and 12 o'clock), as well as between time points in different neighboring cycles (e.g., today's 10 o'clock and tomorrow's 10 o'clock). Taking the time series of temperature changes as an example, the temperature changes at different moments of each day form the daily cycle, and the average temperature changes of each week construct the weekly cycle. Converting the time dimension into two dimensions is conducive to a more precise control of the pattern and trend of temperature change.

As a result, the authors propose the TimesBlock module to implement the relevant functionality, which processes the time series data through the following steps:

1. Base Frequency Extraction: Time series data are analyzed in the frequency domain by Fast Fourier Transform (FFT) to obtain the fundamental frequency components of the data.
2. Time series reshaping: the reorganization of time series data into a two-dimensional variational form based on the main underlying frequencies. This usually means decomposing the time series into multiple frequency components and representing them as a two-dimensional matrix.
3. 2D Convolution: performs a 2D convolution operation on these 2D matrices, which helps in capturing localized patterns and structures in the time series data.
4. Output Reconstruction: the output of the convolution operation is reorganized and summed with the corresponding weights to form the final output.

In the graph shown below, the time series exhibits three cyclical phases (Period 1, Period 2, Period 3). The red color represents the change within the cycle (similar to a ring comparison, e.g., how yesterday changed versus today), and the blue color represents the change between cycles (similar to the concept of year-over-year, e.g., 10:00 today versus 10:00 tomorrow). This time series is converted into three different kinds of two-dimensional data. In each two-dimensional data, each column is used to characterize the change over the cycle, while each row is used to reflect the state of change over the weekly period:

So, how should periodicity be determined in a time series? The Fast Fourier Transform can be used. By applying the fast Fourier transform to the time series, the main periods will be presented as frequency components with high amplitudes. Set a hyperparameter k, and then select only the topk frequency components with the highest amplitude, so that the k major cycles can be obtained. Assuming that k is set to 3, after the fast Fourier transform, the three frequencies with the highest amplitude are 10Hz, 20Hz and 30Hz, and then the periods corresponding to these three frequencies are the major periods. Then the 1D time series is reshaped according to these three cycles, and then processed with 2D convolution and aggregated the results, which can effectively capture the characteristics of the temporal variations of multiple cycles:

TimesBlock module

TimesBlock implements further feature extraction of timing change features, and the implementation of the code is located in the files under the mods file:TimesBlock class. the TimesBlock class is defined as follows:

class TimesBlock():
    def __init__(self, configs):
        super(TimesBlock, self).__init__()
        pass

    def forward(self, x):
        pass

Based on the TimesBlock module, it is possible to build TimesNet models adapted to various types of time-series tasks. The relevant code is located in the files under the models file:TimesNet class. The Model class that implements the TimesNet model is defined as follows:

class Model():
    def __init__(self, configs):
        super(Model, self).__init__()
         = configs
        self.task_name = configs.task_name
        pass

    # Long- and short-term forecasting
    def forecast(self, x_enc, x_mark_enc, x_dec, x_mark_dec):
        ...

    # Missing value filling
    def imputation(self, x_enc, x_mark_enc, x_dec, x_mark_dec, mask):
        pass

    # anomaly detection
    def anomaly_detection(self, x_enc):
        pass
        
    # categorization
    def classification(self, x_enc, x_mark_enc):
        pass

    # inference
    def forward(self, x_enc, x_mark_enc, x_dec, x_mark_dec, mask=None):
        if self.task_name == 'long_term_forecast' or self.task_name == 'short_term_forecast':
            dec_out = (x_enc, x_mark_enc, x_dec, x_mark_dec)
            pass
        if self.task_name == 'imputation':
            dec_out = (
                x_enc, x_mark_enc, x_dec, x_mark_dec, mask)
            pass
        if self.task_name == 'anomaly_detection':
            dec_out = self.anomaly_detection(x_enc)
            pass
        if self.task_name == 'classification':
            dec_out = (x_enc, x_mark_enc)
            pass

The Model class determines which task's processing logic to execute by taking the configs parameter in the initialization function and deciding which task to execute based on the task_name in it. This allows the model to switch the task executed by the model by modifying the config without making major changes to the code structure. At the same time from the above code can be seen, if you need to add a new task in the future, such as regression, you can easily add a new method regression to deal with the relevant logic, greatly increasing the flexibility of the code.

model training

The code related to the training, validation and testing of TimesNet networks is implemented in the exp directory. As an example, the model training process used for the long-time prediction task can be briefly divided into several parts, including data preparation, creation of save paths, initialization, optimizer and loss function selection, training with mixed precision, training loops, validation and early stopping, learning rate tuning, and loading the best model. The relevant code is located atThe file exp/exp_long_term_forecasting.py。

1. Model training with automatic mixing precision (AMP) support: source code located atexp/exp_long_term_forecasting.py。
2. EarlyStopping: the source code is located in theutils/。
3. Optimizer and loss calculation using the Adam optimizer and MSEloss to measure the loss between actual and predicted data: source code located atexp/exp_long_term_forecasting.py
4. Verification and testing: the source code is located atexp/exp_long_term_forecasting.py。
5. dataset selection, providing factory functions to support different dataset reads: the source code is located in thedata_provider/data_factory.py。
6. Data read-in, which reads and divides the dataset according to the selected dataset and the configuration in which the dataset is loaded: the source code is located in thedata_provider/data_loader.py。

Dataset Support

The official TSLiB repository provides downloads of common timing prediction datasets:Google cloudrespond in singingBaidu cloud, and then just put the downloaded dataset into the dataset directory.

1.3 Data sets supported by the TSLiB library

This section focuses on the time-series datasets used in TSLiB. These datasets have been studied by previous researchers in the field of time-series analysis and have been carefully cleaned to conform to a unified format standard. Given the relative immaturity of the time series analysis field, most of the existing datasets rely on web crawling techniques to obtain them, or have some gap with real application scenarios. Specific descriptions of these datasets can be viewed in the TimesNet paper:TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis。

For more descriptions of time-series datasets see:Standardized dataset for time series papersrespond in singingTimesNet replication results。

1.3.1 Long-term forecasting

Long-time prediction is the main task supported by major temporal models. In long-time prediction, the TSLiB library trains the model based on mean square error (MSEloss) loss. Note that the number of features does not contain temporal features, but does contain prediction target features. This is because the long-time prediction model integrated in TSLiB learns the predictor variables as part of the input features during the training process.

data set	Number of data input features	Sample length	time granularity
ECL	321	26304	1 hour
ETTh1/ETTh2	7	17420	1 hour
ETTm1/ETTm2	7	69680	15 minutes.
Exchange_rate	8	7588	1 day
ILI	7	966	1 week
Traffic	862	17544	1 hour
Weather	21	52696	10 minutes.

Note that the prediction target columns of the dataset are all named "OT". This naming was done in order to construct a standardized long-time prediction dataset by unifying the naming of all prediction target columns into a form consistent with the ETT dataset column names.

ECL

The ECL (electricity) dataset covers the hourly electricity consumption records of 321 customers over a four-year period and aims to predict the demand for electricity in different regions. Given the presence of missing values in the data, the last column in the dataset was used as a prediction target. In TSLiB based on theDataset_Customclass to read the dataset.

ETT

The ETT-small Electricity Transformer Temperature (ETT) dataset is mainly applied to the research of time prediction of power system. This dataset covers two years of data from two different regions in China, and consists of two hourly-level datasets (ETTh1, ETTh2) and two 15-minute-level datasets (ETTm1, ETTm2) together. Each data point contains the target value "oil temperature" for the period from July 2016 to July 2018 and six power load characteristics. In TSLiB based onDataset_ETT_hourcap (a poem)Dataset_ETT_minuteclass to read the dataset. The columns are described below:

• date: The date of the record;
• HUFL: High Use Full Load；
• HULL: High Use Less Load；
• MUFL: Middle Use Full Load；
• MULL: Middle Use Less Load；
• LUFL: Low Use Full Load；
• LULL: Low Use Less Load；
• OT: Oil Temperature, predicted target value.

Exchange_rate

The Exchange_rate dataset consists of daily exchange rate data for eight countries, Australia, the UK, Canada, Switzerland, China, Japan, New Zealand, and Singapore, spanning the period from 1990 to 2016. In TSLiB based on theDataset_Customclass to read the dataset.

ILI

The ILI(illness) dataset includes weekly influenza data from the CDC from 2002 to 2020. In TSLiB based on theDataset_Customclass to read the dataset. Each feature column is described below:

• date: The date of the record;
• %WEIGHTED ILI: Country-level weighted ratio;
• %UNWEIGHTED ILI: State-level weighted percentage;
• AGE 0-4: Number of patients under 4 years of age;
• AGE 5-24: Number of patients aged 5 to 24 years;
• ILITOTAL: Total number of patients with influenza-like symptoms;
• PROVIDERS: The number of medical providers reporting data;
• OT: TOTAL PATIENTS, the total number of patients who visited a medical provider.

Traffic

The Traffic dataset contains hourly data records from 862 sensors from the California Department of Transportation between 2016 and 2018. These data detail roadway occupancy as measured by each sensor on San Francisco Bay Area freeways. The roadway occupancy values range from 0 to 1, with 0 indicating that the roadway is completely empty and 1 indicating that the roadway is completely congested, a range of values that reflects how the roadway is actually being used. The last column serves as the feature column. In TSLiB based on theDataset_Customclass to read the dataset.

Weather

The Weather dataset covers 21 key meteorological indicators, including air temperature and humidity. The data was collected from 2020 and recorded at a frequency of every 10 minutes. In TSLiB based on theDataset_Customclass to read the dataset. The columns are described below:

1. p (mbar) / Barometric (millibar): A measure of atmospheric pressure in millibars;
2. T (degC) / Temperature in degrees Celsius: thermodynamic temperature of air in degrees Celsius;
3. Tpot (K) / Potential Temperature in Kelvin: Air temperature in Kelvin after accounting for the effects of humidity;
4. Tdew (degC) / Dew point temperature in degrees Celsius: The temperature in degrees Celsius at which water vapor in air condenses into dew;
5. rh (%) / Relative Humidity (%): The ratio of the water vapor content of the air to the saturated water vapor content, expressed as a percentage;
6. VPmax (mbar) / Maximum Vapor Pressure (mbar): The maximum pressure in millibars that water vapor in air may reach;
7. VPact (mbar) / Actual Vapor Pressure in millibars: The actual pressure of water vapor in the current air in millibars;
8. VPdef (mbar) / Vapor Pressure Difference (mbar): The difference between the actual vapor pressure and the maximum vapor pressure in millibars;
9. sh (g/kg) / specific humidity (g/kg): mass of water vapor per unit mass of air in grams per kilogram;
10. H2OC (mmol/mol) / Water Vapor Concentration (mmol/mol): Molar concentration of water vapor in air in mmol/mol;
11. rho (g/m³) / air density in grams per cubic meter: mass per unit volume of air in grams per cubic meter;
12. wv (m/s) / Horizontal Wind Velocity (m/s): the velocity of the wind in the horizontal direction in meters per second;
13. max. wv (m/s) / Maximum Wind Velocity (m/s): The maximum value of the wind velocity during the recording period in meters per second;
14. wd (deg) / wind direction (deg): the direction from which the wind is coming, expressed as an angle;
15. rain (mm) / precipitation in millimeters: total amount of precipitation in millimeters;
16. raining (s) / rainfall time (s): the duration of rainfall in seconds;
17. SWDR (W/m²) / Direct Solar Radiation (Watts per square meter): Direct solar radiation energy received per unit area in Watts per square meter;
18. PAR (μmol/m²s) / Photosynthetically Active Radiation (μmol/m²s): the effective radiation in which plants photosynthesize, in units of μmol/m²s;
19. max. PAR (μmol/m²s) / Maximum photosynthetically active radiation (μmol/m²s): Maximum value of photosynthetically active radiation in μmol/m²s during the recording period;
20. Tlog (degC) / logarithmic mean temperature in degrees Celsius: the average temperature in degrees Celsius calculated by the logarithmic mean method;
21. OT / CO2 (ppm, I searched for it, not necessarily correct, but about 80% probability of being correct): the unit of concentration of carbon dioxide "parts per million" (parts per million concentration).

1.3.2 Short-term forecasts

The core difference between TSLiB when dealing with short and long time prediction tasks is only the time span of the prediction. Deep learning temporal prediction models are able to generate the entire prediction sequence at once. Therefore, analyzing from the perspective of model structure, there is no significant difference in nature between short-time and long-time prediction, and the main adjustment only involves the setting of the hyperparameter of model output length. For short-time prediction, the symmetric mean absolute percentage error (SMAPE) is used as the benchmark for the calculation of the loss function.
TSLiB uses the M4 dataset to evaluate the ability of the model to make short-term predictions.The M4 dataset is described in detail in:M4The M4 dataset is a collection of multiple time-series data from the 4th Makridakis Forecasting Contest, which consists of time series with different data granularity such as annual, quarterly, monthly, weekly, daily, hourly, etc., and is divided into a training set and a test set. It covers a wide range of fields such as economy, finance, business, society, environment, etc. The main lower data files of the M4 dataset are as follows:

• : Annual data training set
• : Annual data training set
• : Quarterly data training set
• : Quarterly data training set
• : Monthly data training set
• : Monthly data training set
• : Weekly data training set
• : Weekly data training set
• : Daily data training set
• : Daily data training set
• : Hourly data training set
• : Hourly data test set
• : A data set description file with the following meanings for the columns of the file:
  • M4id: ID of the data
  • category: area of data
  • Frequency: the frequency of the data
  • Horizon: the length of the sequence of data
  • SP: Time granularity of data
  • StartingDate: data starting time

1.3.3 Missing value filling

Missing value filling is a task aimed at predicting certain missing values in a time series and is therefore similar to forecasting in a way. The datasets involved in missing value filling Imputation are ECL, ETT, Weather datasets. These datasets have already been mentioned in Long-Term Prediction.The model architecture used to implement Imputation in the TSLiB library is the same as for Long-Term Prediction. However, the difference is that TSLiB masks some of the information in the input data during the training process and subsequently uses the model to predict the masked values in the input data. And the difference between the masked parts is calculated based on the mean square error (MSEloss) loss. This stage does not require the involvement of training labels.

1.3.4 Classification

The TSLib library trains a temporal classification model based on CrossEntropyLoss loss, noting that the input data features do not contain temporal and categorical features. The temporal classification training process is similar to the training process for image classification models.

data set	Number of input data features	Length of training samples	Number of classification categories
EthanolConcentration	3	261 (1751, 3) feature vectors	4
FaceDetection	144	5890 (62, 144) feature vectors	2
Handwriting	3	150 (152,3) eigenvectors	26
Heartbeat	61	204 (405,61) eigenvectors	2
JapaneseVowels	12	270 (29,12) eigenvectors	9
PEMS-SF	963	267 (144,963) feature vectors	7
SelfRegulationSCP1	6	268 (896,6) eigenvectors	2
SelfRegulationSCP2	7	200 eigenvectors of (1152,7)	2
SpokenArabicDigits	13	6599 eigenvectors of (93,13)	10
UWaveGestureLibrary	3	120 eigenvectors of (315,3)	8

EthanolConcentration

The EthanolConcentration dataset is a multivariate time-series dataset that contains raw spectral data from water and ethanol solutions in 44 different real whiskey bottles. These data were used to predict the ethanol concentration in the samples. The ethanol concentrations in the dataset are 35%, 38%, 40% and 45%, where 40% is the minimum legal alcohol content for Scotch whisky. The dataset contains 524 multivariate time series divided into a training set (261 samples) and a test set (263 samples). Each time series consists of 1751 time points where measurements of three variables were recorded. These variables represent spectral intensities at different wavelengths, and the data cover the wavelength range from 226 nm to 1101.5 nm, with a sampling frequency of 0.5 nm.The columns of the input dataset include:

1. Time Point: Indicates the time measurement point.
2. Variable 1: Indicates the first spectral intensity measurement.
3. Variable 2: Indicates the second spectral intensity measurement.
4. Variable 3: Indicates the third spectral intensity measurement.

In addition, there is a numerical vector classes which contains four categories corresponding to the input data, each associated with a different ethanol concentration. Therefore the model classifies the vector with dimensionality dimension of (1751,3) to determine which category it belongs to. In TSLiB based onUEAloaderclass to read the dataset, this class is not the same as the normal temporal data reading class. It is overly verbose in there because the categorical raw data is standardized to ts format and requires too much data processing code.

FaceDetection

The FaceDetection dataset belongs to the facial time-series data, where each group contains 62 observations, each of which in turn has 144 feature points. This dataset is mainly used to distinguish whether the input data represents a human face or a non-face. The reading speed of this dataset is slow. In TSLiB based onUEAloaderclass to read the dataset.

Handwriting

The Handwriting dataset contains handwriting samples of 26 letters with 150 training cases and 850 test cases. The three feature dimensions included in the dataset are accelerometer and gyroscope outputs. In TSLiB based on theUEAloaderclass to read the dataset.

Heartbeat

The Heartbeat dataset is a heartbeat audio monitoring time-series data that is categorized into sound normal and abnormal based on the heartbeat sound. There are 204 training cases and 205 test cases. In TSLiB based onUEAloaderclass to read the dataset.

JapaneseVowels

The JapaneseVowels dataset records speech time series from nine male Japanese speakers. The features are 12 LPC cepstrum coefficients, and there are 270 training cases and 370 test cases. In TSLiB based on theUEAloaderclass to read the dataset.

PEMS-SF

The PEMS-SF data describes the occupancy of different lanes of the San Francisco Bay Area freeways between 0 and 1. Measurements were taken from January 1, 2008 to March 30, 2009, with samples taken every 10 minutes. Each day in the database was treated as a single time series with dimension 963 (number of sensors continuously operating throughout the study period) and length 6x24=144. There are 267 training cases and 173 test cases. In TSLiB based on theUEAloaderclass to read the dataset.

SelfRegulationSCP

The SelfRegulationSCP1 and SelfRegulationSCP2 datasets are datasets for in-depth study of cortical slow potential timing data related to self-regulation. Detailed descriptions of the datasets are given in:A spelling device for the paralysed. In TSLiB based onUEAloaderclass to read the dataset.

SpokenArabicDigits

The SpokenArabicDigits dataset contains time series of Mel Frequency Cepstrum Coefficients (MFCC) corresponding to spoken Arabic digits. A sample consisting of 8800 (10 digits × 10 repetitions × 88 speakers) was collected from 44 male and 44 female native Arabic speakers. Each sample consisted of 13 Mel Frequency Cepstrum Coefficients (MFCCs) to represent 10 spoken Arabic digits. There are 6599 training cases and 2199 test cases. In TSLiB based on theUEAloaderclass to read the dataset.

UWaveGestureLibrary

The UWaveGestureLibrary dataset is a collection of time-series data about gesture movements. In this case, the time-series data is composed of X, Y, and Z coordinates corresponding to each motion, and each sequence has a length of 315. The dataset records a total of 8 different gestures. There are 120 training cases and 320 test cases. In TSLiB based onUEAloaderclass to read the dataset.

1.3.5 Anomaly detection

Anomaly detection is trained in a similar way to missing value filling Imputation, i.e., the model is trained to reconstruct the input data and MSEloss is used to compute the reconstruction error. In the testing phase, the model tries to reconstruct the input data. If the reconstruction error is above a certain threshold, the data points are considered anomalous. The threshold is usually set to the high percentile of the training data so that only those data points with very high reconstruction errors are labeled as anomalous. The predicted anomaly labels are then obtained and compared with the true anomaly labels to calculate evaluation metrics such as accuracy.

data set	Number of data input features	sequence length
MSL	55	100
PSM	25	100
SMAP	25	100
SMD	38	100
SWaT	51	100

MSL

The MSL (Mars Science Laboratory rover) dataset contains anomalies from the Mars Curiosity rover. The input is probe data from 55 channels. The training set has no label and the test set has a status label. the status label is either 0 or 1 (0 means that the signal was not sent, while a 1 signal means that it was sent). More information about the dataset can be found at/khundman/telemanom.. There are a total of 58,317 training cases, 11,664 validation cases (from 20% data at the end of the training data) and 73,729 test cases. In TSLiB based on theMSLSegLoaderclass to read the dataset. During training, 100 consecutive sequences are randomly extracted at a time for data reconstruction.

PSM

The PSM dataset was collected from within multiple application server nodes at eBay with 132481 training cases, 26398 validation cases (from 20% of the data at the end of the training data) and 87841 test cases. In TSLiB based on thePSMSegLoaderclass to read the dataset.

SMAP

The SMAP dataset is a spacecraft dataset with 25 channels of probe data as input. There are 135,084 training cases, 26,938 validation cases (from 20% of the data at the end of the training data) and 427,518 test cases. In TSLiB based on theSMAPSegLoaderclass to read the dataset.

SMD

SMD (Server Machine Dataset, Server Machine Datase) consists of 38 sensor data. There are 708,405 training cases, 141,681 validation cases (from 20% data at the end of the training data) and 708,420 test cases. In TSLiB based on theSMDSegLoaderclass to read the dataset.

SWaT

The SWaT dataset contains timing anomaly data from sensors in water treatment systems. These data features cover information from 51 sensors and actuators. The dataset records 11 consecutive days of operation, with the first 7 days of normal operation and the last 4 days simulating exposure to 41 staged attacks. The temporal granularity of the data is once per second. There are 495,000 training cases, 99,000 validation cases (from the 20% data at the end of the training data) and 449,919 test cases. In TSLiB based on theSWATSegLoaderclass to read the dataset.

1.4 Model training

The TSLiB library provides training scripts for different models for various timing tasks, as described in the TSLiB project repository'sscriptsFolder. The dataset ETTh1 is used as an example to see how TimesNet performs in a long-time prediction task:

model_name=TimesNet

python -u  \
  --task_name long_term_forecast \
  --is_training 1 \
  --root_path ./dataset/ETT-small/ \
  --data_path  \
  --model_id ETTh1_96_96 \
  --model $model_name \
  --data ETTh1 \
  --features M \
  --seq_len 96 \
  --label_len 48 \
  --pred_len 96 \
  --e_layers 2 \
  --d_layers 1 \
  --factor 3 \
  --enc_in 7 \
  --dec_in 7 \
  --c_out 7 \
  --d_model 16 \
  --d_ff 32 \
  --des 'Exp' \
  --itr 1 \
  --top_k 5 

Different training tasks and different models correspond to different training parameters. The introduction of these training parameters can be found by using the argparse parameter in the project file to find the meaning and introduction of all parameters:/thuml/Time-Series-Library/blob/main/#L19-L135。

The relevant scripts can be executed by running the following code in a Linux environment. Note that the scripts for the different models contain multiple training options, which need to be extracted and used with the single run option:

./scripts/long_term_forecast/ETT_script/TimesNet_ETTh1.sh

During the training process, the dataset losses for the training, validation and test sets are output. Many codebases in the deep learning field generally tend to print the loss data of the test set in real time during the training process. The purpose of this practice is to quickly tune the parameters while avoiding testing the datasets one by one. However, this is not the right thing to do in an engineering project because of test data leakage.

Epoch: 2, Steps: 265 | Train Loss: 0.3978408 Vali Loss: 0.8001348 Test Loss: 0.3954313

Immediately after the model training is completed, model testing will be performed to evaluate the performance of the model on the test set. Note that TSLiB does not provide code for model inference, and you need to write the relevant code yourself:

print('>>>>>>>start training : {}>>>>>>>>>>>>>>>>>>>>>>>>>>'.format(setting))
(setting)

print('>>>>>>>testing : {}<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<'.format(setting))
(setting)
.empty_cache()

The trained model weights are saved in the checkpoints folder. The test set detailed results are saved in the results folder. There are also detailed visualization results saved in PDF format in test_results folder as shown below.

In the above graph, the horizontal axis indicates the length of the sequence and the vertical axis shows the predicted results. Since the data is normalized during processing, both the predicted and actual results are negative. It is important to note that the TimesNet model also normalizes the input data during the training and inference phases.

2 Reference

• Time-Series-Library
• TimesNet: a time-series base model with five leading tasks such as prediction, filling, and classification
• TimesNet_tutorial.ipynb
• TimesBlock class
• TimesNet class
• TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis
• Standardized dataset for time series papers
• TimesNet replication results
• M4
• A spelling device for the paralysed