Improved Spatial-temporal Neural Networks and the Application

Multivariate time series (MTS) forecasting is the study of analyzing the change of multiple variables over time, whose major objective is to predict the future based on existing historical data records. Such tasks are almost involved in the natural and social sciences in various fields and have very broad development prospects, such as sociology, engineering, economics, physics, etc. Therefore, it is also one of the hottest topics with significant research value and strong practicability in recent years. In practice, one variable is not only determined by its historical value but also influenced by other variables with which it has relationships. Overall, there are three main types of dependencies between different samples: temporal dependency, spatial dependency, and temporal-spatial dependency. Related research mainly focuses on how to make the model more accurately capture those dependencies, especially implicit ones. In recent years, due to the improvement of computing power and benefiting from the large amount of data generated by internet-related technologies, the performance of the data-driven model has made a spurt of progress and become one of the most eye-catching fields worldwide. Thus, introducing deep learning methods that are the most popular data- driven models into the MTS forecasting task has become the mainstream research direction instantly. In the thesis, we first review the previous work and summarize the main challenges faced in current research. Then we design two novel models according to known limitations to address those challenges. The first one is named Fully Spatial-Temporal Graph Recurrent Convolutional Neural Networks (FGRCNN), which learns the adjacent matrix using a novel adaptive learning algorithm and extracts dependencies based on a sandwich-like framework. Besides, we also propose two novel structures: Gated Convolutional ResNet (GCR) and Gated sequential convolutional (GSC) block to improve the model’s performance. The second model is Attention-Based Spatial-Temporal Fusion Neural Networks (ASTFNN), which contains an improved spatial-temporal attention mechanism to capture all kinds of dependencies simultaneously. And we also propose a dynamic graph generation algorithm to make the model generate unique graph structures for each individual input. After that, we train the model with several real-world datasets. Experimental results demonstrate that our method achieves state-of-the-art (SOTA) performance compared with other competitive baseline approaches. Ablation experiments are also conducted to prove the positive contribution of each proposed sub-algorithm. Finally, the conclusions are given, and future research topics are also summarized.