Handwritten Mathematical Expression Recognition using Deep Learning Techniques
Keywords:
Handwritten Mathematical Expression Recognition, Convolutional Neural Networks (CNNs), Deep Learning, Image Processing, OCR, Symbol Classification, Eval Function, Neural Networks, Educational Technology, Real-Time EvaluationAbstract
The accurate recognition and computational evaluation of handwritten mathematical expressions present a significant challenge in the domain of intelligent systems and digital education. This complexity is primarily due to the diverse nature of human handwriting and the inherently two-dimensional structure of mathematical notation, which traditional Optical Character Recognition (OCR) systems fail to interpret reliably. To address these limitations, this study introduces a deep learning-based framework employing Convolutional Neural Networks (CNNs) for the classification of individual handwritten symbols. The system is trained on a curated dataset of over 96,000 grayscale images encompassing 13 classes, including numeric digits and basic arithmetic operators. After classification, the identified symbols are reconstructed into complete expressions and evaluated using a programmatic method based on Python’s eval() function. The model achieves a training accuracy of 99.55%, demonstrating its efficacy in symbol recognition. Preprocessing techniques such as grayscale conversion, thresholding, contour extraction, and image normalization ensure consistent and high-quality input. The system’s modular design and low computational overhead make it suitable for real-world deployment, including on embedded and mobile platforms. This work lays a foundation for scalable, efficient, and accurate recognition of handwritten mathematical content, contributing to advancements in educational technologies and human-computer interaction
Downloads
Metrics
References
Chollet, F. (2017). "Xception: Deep Learning with Depthwise Separable Convolutions." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1251–1258. DOI: 10.1109/CVPR.2017.195
Deng, Y., Kanervisto, A., Ling, J., & Rush, A. M. (2017). "Image-to-Markup Generation with Coarse-to-Fine Attention." Proceedings of the 34th International Conference on Machine Learning (ICML). Available: https://arxiv.org/abs/1706.01006
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press. Available: https://www.deeplearningbook.org/
Graves, A., & Schmidhuber, J. (2009). "Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks." Advances in Neural Information Processing Systems (NIPS), pp. 545–552. Available: https://papers.nips.cc/paper/2008/file/96e3a2c9f5c0d4d9e6a7755b3ff4bf2b-Paper.pdf
Guo, Y., Chen, S., & Li, J. (2017). "Multi-Scale Attention Model for Handwritten Mathematical Expression Recognition." 2017 IEEE International Conference on Computer Vision (ICCV) Workshops, pp. 40–46. DOI: 10.1109/ICCVW.2017.40
LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). "Gradient-based Learning Applied to Document Recognition." Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324. DOI: 10.1109/5.726791
LeCun, Y., Cortes, C., & Burges, C. J. C. (1998). "MNIST Handwritten Digit Database." Available: http://yann.lecun.com/exdb/mnist/
Matsakis, N. E., & Zanibbi, R. (2014). "Recognizing Handwritten Mathematical Expressions with Tree Transducers and Symbol Context." 2014 14th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 172–177. DOI: 10.1109/ICFHR.2014.172
Mouchère, H., Zanibbi, R., Viard-Gaudin, C., & Ah-Soon, C. (2016). "ICFHR 2016 CROHME Competition: Handwritten Mathematical Expression Recognition." 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 492–497. DOI: 10.1109/ICFHR.2016.0124
Shi, B., Bai, X., & Yao, C. (2017). "An End-to-End Trainable Neural Network for Image-Based Sequence Recognition and Its Application to Scene Text Recognition." IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 11, pp. 2298–2304. DOI: 10.1109/TPAMI.2016.2646371
Shi, B., Wang, X., Lyu, P., & Yao, C. (2020). "ASTER: An Attentional Scene Text Recognizer with Flexible Rectification." IEEE Transactions on Pattern Analysis and Machine Intelligence. DOI: 10.1109/TPAMI.2019.2957516
Simonyan, K., & Zisserman, A. (2015). "Very Deep Convolutional Networks for Large-Scale Image Recognition." International Conference on Learning Representations (ICLR). Available: https://arxiv.org/abs/1409.1556
Szegedy, C., et al. (2015). "Going Deeper with Convolutions." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9. DOI: 10.1109/CVPR.2015.7298594
Tappert, C. C., Suen, C. Y., & Wakahara, T. (1990). "The State of the Art in Online Handwriting Recognition." IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 12, no. 8, pp. 787–808. DOI: 10.1109/34.57481
Tan, M., & Le, Q. (2019). "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks." Proceedings of the 36th International Conference on Machine Learning (ICML), pp. 6105–6114. Available: https://arxiv.org/abs/1905.11946
Tang, J., Deng, C., Huang, G., & Zhao, L. (2018). "Deep Learning-Based Handwritten Mathematical Symbol Recognition: A Survey." Journal of Computer Science and Technology, vol. 33, no. 4, pp. 675–688. DOI: 10.1007/s11390-018-1855-9
You, S., & Zanibbi, R. (2017). "Recognition of Mathematical Expressions with Nested 2D Structures." Pattern Recognition Letters, vol. 95, pp. 18–25. DOI: 10.1016/j.patrec.2017.02.014
Zanibbi, R., & Blostein, D. (2012). "Recognition and Retrieval of Mathematical Expressions." International Journal on Document Analysis and Recognition (IJDAR), vol. 15, pp. 331–357. DOI: 10.1007/s10032-012-0184-7
Zanibbi, R., Mouchère, H., & Garain, U. (2017). "Advances in Handwritten Mathematical Expression Recognition: The CROHME Competitions and Beyond." In Handbook of Document Image Processing and Recognition. Springer. DOI: 10.1007/978-1-4471-5104-6_38
Zhang, Z., Tang, J., Wang, X., & Qian, X. (2017). "Handwritten Mathematical Expression Recognition Using Multi-Scale Attention with Dense Encoder." AAAI Conference on Artificial Intelligence, pp. 1–8. Available: https://arxiv.org/abs/1712.01054
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
You are free to:
- Share — copy and redistribute the material in any medium or format
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
Terms:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
