Evaluating Machine Learning and Deep Learning Models for Predictive Maintenance: A Study Using the AI4I 2020 Dataset
Keywords:
Predictive Maintenance, AI4I 2020, Machine Learning, Deep Learning, XGBoost, Random Forest, SVM, Neural Networks, Industrial IoT,, F1-ScoreAbstract
Predictive maintenance leverages data-driven approaches to foresee equipment failures and reduce downtime in industrial settings. This study evaluates the performance of several machine learning (ML) and deep learning (DL) models on the AI4I 2020 synthetic dataset, which simulates a milling machine's operational conditions and failure types. Models including Random Forest, Support Vector Machine (SVM), XGBoost, and a deep neural network were assessed using standard classification metrics such as Accuracy, F1-Score, Precision, and Recall. Surprisingly high F1-scores, often exceeding 0.995, were achieved across all classifiers and failure types. This exceptional performance is attributed to the dataset's high quality, clear feature-label relationships, and absence of noise. We analyze the implications of such results, highlighting potential limitations in model generalization to real-world scenarios. The study underscores the importance of dataset characteristics, model selection, and validation strategies in predictive maintenance applications. Practical insights and guidelines are provided to support the deployment of such models in industrial environments, with emphasis on validation against real-world conditions and robustness testing.
Downloads
References
Lee, J., Bagheri, B., & Kao, H.A. (2015). A Cyber-Physical Systems architecture for Industry 4.0-based manufacturing systems. Manufacturing Letters, 3, 18–23.
Zhang, W., Yang, D., & Wang, H. (2019). Data-driven methods for predictive maintenance of industrial equipment: A survey. IEEE Systems Journal, 13(3), 2213–2227.
Carvalho, T. P., et al. (2019). A systematic literature review of machine learning methods applied to predictive maintenance. Computers & Industrial Engineering, 137, 106024.
Lee, J., Davari, H., Singh, J., & Pandhare, V. (2018). Industrial Artificial Intelligence for industry 4.0-based manufacturing systems. Manufacturing Letters, 18, 20–23.
AI4I 2020 Dataset. https://archive.ics.uci.edu/ml/datasets/AI4I+2020+Predictive+Maintenance+Dataset
Bousdekis, A., Magoutas, B., Apostolou, D., & Mentzas, G. (2019). Review, analysis and synthesis of predictive maintenance methods, practices and tools. Journal of Intelligent Manufacturing, 30(3), 985–1003.
Widodo, A., & Yang, B. S. (2007). Support vector machine in machine condition monitoring and fault diagnosis. Mechanical Systems and Signal Processing, 21(6), 2560–2574.
Zhang, Z., Lim, C.P., Nahavandi, S., & Dou, H. (2019). Deep learning-based fault diagnosis using adversarial domain adaptation. IEEE Access, 7, 110895–110906.
Zhao, R., Yan, R., Chen, Z., Mao, K., Wang, P., & Gao, R.X. (2019). Deep learning and its applications to machine health monitoring. Mechanical Systems and Signal Processing, 115, 213–237.
Malhi, A., & Gao, R.X. (2004). PCA-based feature selection scheme for machine defect classification. IEEE Transactions on Instrumentation and Measurement, 53(6), 1517–1525.
Zonta, T., da Costa, C.A., da Rosa Righi, R., de Lima, M.J., da Trindade, E.S., & Li, G.P. (2020). Predictive maintenance in the Industry 4.0: A systematic literature review. Computers & Industrial Engineering, 150, 106889.
Zhang, C., Zhang, Y., & Liu, W. (2017). A hybrid method for gear fault diagnosis using vibration signal. Measurement, 103, 52–60.
Sikorska, J.Z., Hodkiewicz, M., & Ma, L. (2011). Prognostic modelling options for remaining useful life estimation by industry. Mechanical Systems and Signal Processing, 25(5), 1803–1836.
Jardine, A.K.S., Lin, D., & Banjevic, D. (2006). A review on machinery diagnostics and prognostics implementing condition-based maintenance. Mechanical Systems and Signal Processing, 20(7), 1483–1510.
Sun, Y., Zhang, W., Tang, Y., & Li, D. (2021). Federated learning with differential privacy for decentralized fault diagnosis. IEEE Transactions on Industrial Informatics, 17(3), 2088–2097.
Arrieta, A.B., et al. (2020). Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion, 58, 82–115.
Pan, S.J., & Yang, Q. (2010). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345–1359.
Lundberg, S.M., & Lee, S.I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30.
Medjaher, K., & Zerhouni, N. (2012). Data-driven prognostics based on health indicator construction: Application to PRONOSTIA’s case study. The International Journal of Prognostics and Health Management, 3(1).
Ramasso, E., & Gouriveau, R. (2014). Prognostics in switching systems: Evidential Markovian classification of real-time neuroevolving recurrent models. Annual Conference of the Prognostics and Health Management Society.
Susto, G.A., Schirru, A., Pampuri, S., McLoone, S., & Beghi, A. (2015). Machine learning for predictive maintenance: A multiple classifier approach. IEEE Transactions on Industrial Informatics, 11(3), 812–820.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
You are free to:
- Share — copy and redistribute the material in any medium or format
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
Terms:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
