Multivariate Temporal Convolutional Network: A Deep Neural Networks Approach for Multivariate Time Series Forecasting

Renzhuo Wan; Shuping Mei; Jun Wang; Min Liu; Fan Yang

doi:10.3390/electronics8080876

journal article Open Access Aug 07, 2019

Multivariate Temporal Convolutional Network: A Deep Neural Networks Approach for Multivariate Time Series Forecasting

Renzhuo Wan

Electronics Vol. 8 No. 8 pp. 876 · MDPI AG

View at Publisher Save 10.3390/electronics8080876

Abstract

Multivariable time series prediction has been widely studied in power energy, aerology, meteorology, finance, transportation, etc. Traditional modeling methods have complex patterns and are inefficient to capture long-term multivariate dependencies of data for desired forecasting accuracy. To address such concerns, various deep learning models based on Recurrent Neural Network (RNN) and Convolutional Neural Network (CNN) methods are proposed. To improve the prediction accuracy and minimize the multivariate time series data dependence for aperiodic data, in this article, Beijing PM2.5 and ISO-NE Dataset are analyzed by a novel Multivariate Temporal Convolution Network (M-TCN) model. In this model, multi-variable time series prediction is constructed as a sequence-to-sequence scenario for non-periodic datasets. The multichannel residual blocks in parallel with asymmetric structure based on deep convolution neural network is proposed. The results are compared with rich competitive algorithms of long short term memory (LSTM), convolutional LSTM (ConvLSTM), Temporal Convolution Network (TCN) and Multivariate Attention LSTM-FCN (MALSTM-FCN), which indicate significant improvement of prediction accuracy, robust and generalization of our model.

Topics

No keywords indexed for this article. Browse by subject →

References

37

[1]

Seyed, B.L., and Behrouz, M. (2008, January 20–22). Comparison between ANN and Decision Tree in Aerology Event Prediction. Proceedings of the International Conference on Advanced Computer Theory & Engineering, Phuket, Thailand.

[2]

Simmonds, J., Gómez, J.A., and Ledezma, A. (2017). Data Preprocessing to Enhance Flow Forecasting in a Tropical River Basin. Engineering Applications of Neural Networks, Springer. 10.1007/978-3-319-65172-9_36

[3]

Mohamad "Artificial intelligence for the prediction of water quality index in groundwater systems" Model. Earth Syst. Environ. (2016) 10.1007/s40808-015-0063-9

[4]

Amato "Multimedia summarization using social media content" Multimed. Tools Appl. (2018) 10.1007/s11042-017-5556-2

[5]

Kadir "Modeling and prediction of Turkey’s electricity consumption using Artificial Neural Networks" Energy Convers. Manag. (2009) 10.1016/j.enconman.2009.06.016

[6]

Wu, Y., José, M.H., and Ghahramani, Z. (2013). Dynamic Covariance Models for Multivariate Financial Time Series. arXiv.

[7]

Yu, R., Li, Y., Shahabi, C., Demiryurek, U., and Liu, Y. (2017, January 27–29). Deep learning: A generic approach for extreme condition traffic forecasting. Proceedings of the 2017 SIAM International Conference on Data Mining, Houston, TX, USA. 10.1137/1.9781611974973.87

[8]

Akaike "Fitting autoregressive models for prediction" Ann. Inst. Stat. Math. (1969) 10.1007/bf02532251

[9]

Frigola, R., and Rasmussen, C.E. (2013, January 10–13). Integrated pre-processing for bayesian nonlinear system identification with gaussian processes. Proceedings of the IEEE Conference on Decision and Control, Florence, Italy. 10.1109/cdc.2013.6760734

[10]

Alom, M., Tha, T., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M., Hasan, M., Essen, B., Awwal, A., and Asari, V. (2019). A State-of-the-Art Survey on Deep Learning Theory and Architectures. Electronics, 8. 10.3390/electronics8030292

[11]

Liu, L., Finch, A.M., Utiyama, M., and Sumita, E. (2016, January 12–17). Agreement on Target-Bidirectional LSTMs for Sequence-to-Sequence Learning. Proceedings of the Thirtieth Aaai Conference on Artificial Intelligence, Phoenix, AZ, USA. 10.1609/aaai.v30i1.10327

[12]

Gehring, J., Auli, M., Grangier, D., Yarats, D., and Dauphin, Y.N. (2017). Convolutional Sequence to Sequence Learning. arXiv.

[13]

Dilated Residual Networks

Fisher Yu, Vladlen Koltun, Thomas Funkhouser

2017 IEEE Conference on Computer Vision and Patter... 10.1109/cvpr.2017.75

[14]

He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.

[15]

ARIMA forecasting of primary energy demand by fuel in Turkey

Volkan Ş. Ediger, Sertaç Akar

Energy Policy 2007 10.1016/j.enpol.2006.05.009

[16]

Rojas "Soft-computing techniques and ARMA model for time series prediction" Neurocomputing (2008) 10.1016/j.neucom.2007.07.018

[17]

Kilian "New introduction to multiple time series analysis" Econ. Rec. (2006)

[18]

Sapankevych "Time Series Prediction Using Support Vector Machines: A Survey" IEEE Comput. Intell. Mag. (2009) 10.1109/mci.2009.932254

[19]

Hamidi "Application of random forest time series, support vector regression and multivariate adaptive regression splines models in prediction of snowfall (a case study of Alvand in the middle Zagros, Iran)" Theor. Appl. Climatol. (2018) 10.1007/s00704-017-2300-9

[20]

Lima "Climate informed monthly streamflow forecasts for the Brazilian hydropower network using a periodic ridge regression model" J. Hydrol. (2010) 10.1016/j.jhydrol.2009.11.016

[21]

Li "Forecasting macroeconomic time series: LASSO-based approaches and their forecast combinations with dynamic factor models" Int. J. Forecast. (2014) 10.1016/j.ijforecast.2014.03.016

[22]

Long Short-Term Memory

Sepp Hochreiter, Jürgen Schmidhuber

Neural Computation 1997 10.1162/neco.1997.9.8.1735

[23]

Shi, X., Chen, Z., Wang, H., Yeung, D.Y., Wong, W.K., and Woo, W.C. (2015, January 7–12). Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting. Proceedings of the Neural Information Processing Systems Conference, Montreal, QC, Canada.

[24]

Karim "Multivariate LSTM-FCNs for Time Series Classification" Neural Netw. (2019) 10.1016/j.neunet.2019.04.014

[25]

The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis

Norden E. Huang, Zheng Shen, Steven R. Long et al.

Proceedings of the Royal Society A: Mathematical,... 1998 10.1098/rspa.1998.0193

[26]

Wu "Ensemble empirical mode decomposition: A noise-assisted data analysis method" Adv. Adapt. Data Anal. (2009) 10.1142/s1793536909000047

[27]

Wang, J., Wang, Z., Li, J., and Wu, J. (2018, January 19–23). Multilevel Wavelet Decomposition Network for Interpretable Time Series Analysis. Proceedings of the 24th ACM SIGKDD International Conference, London, UK. 10.1145/3219819.3220060

[28]

Dragomiretskiy "Variational Mode Decomposition" IEEE Trans. Signal Process. (2014) 10.1109/tsp.2013.2288675

[29]

Bai, S., Kolter, J.Z., and Koltun, V. (2018). An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling. arXiv.

[30]

Dou "Hybrid model for renewable energy and loads prediction based on data mining and variational mode decomposition" IET Gener. Transm. Distrib. (2018) 10.1049/iet-gtd.2017.1476

[31]

Searching for MobileNetV3

Andrew Howard, Mark Sandler, Bo Chen et al.

2019 IEEE/CVF International Conference on Computer... 10.1109/iccv.2019.00140

[32]

Cai, H., Zhu, L., and Han, S. (2018). ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware. arXiv.

[33]

Elsken, T., Metzen, J.H., and Hutter, F. (2018). Neural Architecture Search: A Survey. arXiv. 10.1007/978-3-030-05318-5_3

[34]

Nair, V., and Hinton, G. (2010, January 21–24). Rectified linear units improve restricted Boltzmann machines. Proceedings of the 27th International Conference on International Conference on Machine Learning, Haifa, Israel.

[35]

Kingma, D., and Ba, J. (2014). Adam:A method for stochastic optimization. arXiv.

[36]

Sutskever, I., Martens, J., Dahl, G.E., and Hinton, G. (2013, January 16–21). On the importance of initialization and momentum in deep learning. Proceedings of the 30th International Conference on International Conference on Machine Learning, Atlanta, GA, USA.

[37]

Ioffe, S., and Szegedy, C. (2015, January 6–11). Batch normalization: Accelerating deep network training by reducing internal covariate. Proceedings of the 32nd International Conference on Machine Learning, Lille, France.

Cited By

302

Prediction of tunnel surrounding rock deformation using CNN-BiLSTM-attention model incorporating a novel index: Excavation factor

Zhi-Qiang Li, Yanjie Yin · 2025

Tunnelling and Underground Space Te...

A Structured Study of Multivariate Time-Series Distance Measures

Jens E. d'Hondt, Fan Yang · 2025

Proceedings of the ACM on Managemen...

Selected Topics in Time Series Forecasting: Statistical Models vs. Machine Learning

Dag Tjøstheim · 2025

Entropy

Novel Speech-Based Emotion Climate Recognition in Peers’ Conversations Incorporating Affect Dynamics and Temporal Convolutional Neural Networks

Ghada Alhussein, Mohanad Alkhodari · 2025

IEEE Access

Weibull analysis of ceramics and related materials: A review

Obinna Anayo Osuchukwu, Abdu Salihi · 2024

Heliyon

Advance prediction of coastal groundwater levels with temporal convolutional and long short-term memory networks

Xiaoying Zhang, Fan Dong · 2023

Hydrology and Earth System Sciences

Deep Learning for Time Series Forecasting: Advances and Open Problems

Angelo Casolaro, Vincenzo Capone · 2023

Information

A Hybrid Prediction Method for Realistic Network Traffic With Temporal Convolutional Network and LSTM

Jing Bi, Xiang Zhang · 2022

IEEE Transactions on Automation Sci...

Short-Term Load Forecasting Using Channel and Temporal Attention Based Temporal Convolutional Network

Xianlun Tang, Hongxu Chen · 2022

Electric Power Systems Research

Multivariable Time Series Forecasting for Urban Water Demand Based on Temporal Convolutional Network Combining Random Forest Feature Selection and Discrete Wavelet Transform

Jun Guo, Hui Sun · 2022

Water Resources Management

Metrics

302

Citations

37

References

Details

Published: Aug 07, 2019
Vol/Issue: 8(8)
Pages: 876
License: View

Authors

R

Renzhuo Wan

Nano-Optical Material and Storage Device Research Center, School of Electronic and Electrical Engineering, Wuhan Textile University, Wuhan 430200, China

S

Shuping Mei

Nano-Optical Material and Storage Device Research Center, School of Electronic and Electrical Engineering, Wuhan Textile University, Wuhan 430200, China

J

Jun Wang

Nano-Optical Material and Storage Device Research Center, School of Electronic and Electrical Engineering, Wuhan Textile University, Wuhan 430200, China

M

Min Liu

State Key Laboratory of Powder Metallurgy, School of Physics and Electronics, Central South University, Changsha 410083, China

F

Fan Yang

Nano-Optical Material and Storage Device Research Center, School of Electronic and Electrical Engineering, Wuhan Textile University, Wuhan 430200, China

Funding

National Natural Science Foundation of China Award: 11505130; 21872142

Project of Innovation-Driven Plan in Central South University Award: 2017CX003

State Key Laboratory of Powder Metallurgy, Shenzhen Science and Technology Innovation Project Award: JCYJ20180307151313532

Cite This Article

Renzhuo Wan, Shuping Mei, Jun Wang, et al. (2019). Multivariate Temporal Convolutional Network: A Deep Neural Networks Approach for Multivariate Time Series Forecasting. Electronics, 8(8), 876. https://doi.org/10.3390/electronics8080876

Multivariate Temporal Convolutional Network: A Deep Neural Networks Approach for Multivariate Time Series Forecasting

You May Also Like