Enhanced Temporal Convolutional Networks for Robust, Efficient Time Series Classification
DOI:
https://doi.org/10.52783/jns.v14.4003Keywords:
Deep learning, temporal convolutional networks, time series classification, dilated convolutions, residual connectionsAbstract
Time interdependence and different sequence lengths make time series data categorization still a difficult problem in machine learning. This article offers a better Temporal Convolutional Network (TCN) design that uses dilated causal convolutions and residual connections to solve these issues. Across many benchmark datasets, we methodically contrast our approach with conventional techniques like Long Short-Term Memory (LSTM) networks, Gated Recurrent Units (GRUs), and 1D Convolutional Neural Networks (CNNs). Experimental findings show that our TCN implementation outperforms traditional methods in terms of classification accuracy, computational efficiency, and resilience to varied sequence lengths. Along with ablation experiments confirming our design choices, we offer thorough examination of architectural options like kernel size, dilation factors, and residual block design. Aiming for uses from healthcare monitoring to industrial sensor data analysis, the suggested architecture has specific strength in encapsulating multivariate time series data's short- and long-term dependency.
Downloads
References
Z. Cui, W. Chen, and Y. Chen, "Multi-scale convolutional neural networks for time series classification,"arXiv preprint arXiv:1603.06995, 2016.
H. Sakoe and S. Chiba, "Dynamic programming algorithm optimization for spoken word recognition,"IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 26, no. 1, pp. 43-49, 1978.
H. Deng, G. Runger, E. Tuv, and M. Vladimir, "A time series forest for classification and feature extraction,"Information Sciences, vol. 239, pp. 142-153, 2013.
H. I. Fawaz, G. Forestier, J. Weber, L. Idoumghar, and P. A. Muller, "Deep learning for time series classification: a review,"Data Mining and Knowledge Discovery, vol. 33, no. 4, pp. 917-963, 2019.
S. Hochreiter and J. Schmidhuber, "Long short-term memory,"Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997.
K. Cho et al., "Learning phrase representations using RNN encoder-decoder for statistical machine translation,"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014, pp. 1724-1734.
Z. C. Lipton, J. Berkowitz, and C. Elkan, "A critical review of recurrent neural networks for sequence learning,"arXiv preprint arXiv:1506.00019, 2015.
Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning,"Nature, vol. 521, no. 7553, pp. 436-444, 2015.
S. Bai, J. Z. Kolter, and V. Koltun, "An empirical evaluation of generic convolutional and recurrent networks for sequence modeling,"arXiv preprint arXiv:1803.01271, 2018.
L. Ye and E. Keogh, "Time series shapelets: a new primitive for data mining,"Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2009, pp. 947-956.
J. Lines and A. Bagnall, "Time series classification with ensembles of elastic distance measures,"Data Mining and Knowledge Discovery, vol. 29, no. 3, pp. 565-592, 2015.
J. Lines, S. Taylor, and A. Bagnall, "HIVE-COTE: The hierarchical vote collective of transformation-based ensembles for time series classification,"IEEE International Conference on Data Mining (ICDM), 2016, pp. 1041-1046.
A. Graves and J. Schmidhuber, "Framewise phoneme classification with bidirectional LSTM and other neural network architectures,"Neural Networks, vol. 18, no. 5-6, pp. 602-610, 2005.
Z. Cui, W. Chen, and Y. Chen, "Multi-scale convolutional neural networks for time series classification,"arXiv preprint arXiv:1603.06995, 2016.
Z. Wang, W. Yan, and T. Oates, "Time series classification from scratch with deep neural networks: A strong baseline,"International Joint Conference on Neural Networks (IJCNN), 2017, pp. 1578-1585.
M. F. Stollenga, W. Byeon, M. Liwicki, and J. Schmidhuber, "Parallel multi-dimensional LSTM, with application to fast biomedical volumetric image segmentation,"Advances in Neural Information Processing Systems, 2015, pp. 2998-3006.
A. Vaswani et al., "Attention is all you need,"Advances in Neural Information Processing Systems, 2017, pp. 5998-6008.
Y. Chen, Y. Kang, Y. Chen, and Z. Wang, "Attention augmented temporal convolutional network for stock trend prediction,"IEEE International Conference on Tools with Artificial Intelligence (ICTAI), 2019, pp. 1210-1217.
K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition,"IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770-778.
A. van den Oord et al., "WaveNet: A generative model for raw audio,"arXiv preprint arXiv:1609.03499, 2016.
C. Lea, M. D. Flynn, R. Vidal, A. Reiter, and G. D. Hager, "Temporal convolutional networks for action segmentation and detection,"IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 156-165.
H. I. Fawaz, G. Forestier, J. Weber, L. Idoumghar, and P. A. Muller, "Deep neural network ensembles for time series classification,"International Joint Conference on Neural Networks (IJCNN), 2019, pp. 1-6.
X. Wang, H. Chen, A. Terzis, and S. Lin, "STFT-Net: Adaptive temporal convolutional neural network for time series classification,"IEEE International Conference on Big Data (Big Data), 2019, pp. 183-192.
A. Bagnall et al., "The UCR time series classification archive,"arXiv preprint arXiv:1810.07758, 2018.
D. Anguita, A. Ghio, L. Oneto, X. Parra, and J. L. Reyes-Ortiz, "A public domain dataset for human activity recognition using smartphones,"European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, 2013.
R. Chavarriaga et al., "The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition,"Pattern Recognition Letters, vol. 34, no. 15, pp. 2033-2042, 2013.
H. I. Fawaz, B. Lucas, G. Forestier, C. Pelletier, D. F. Schmidt, J. Weber, G. I. Webb, L. Idoumghar, P. A. Muller, and F. Petitjean, "InceptionTime: Finding AlexNet for time series classification,"Data Mining and Knowledge Discovery, vol. 34, no. 6, pp. 1936-1962, 2020.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
You are free to:
- Share — copy and redistribute the material in any medium or format
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
Terms:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
