Thermal Image-Based Multi-Class Semantic Segmentation for Autonomous Vehicle Navigation in Restricted Environments
Abstract
Technological advancements have propelled the development of environmentally friendly transportation, with autonomous vehicles (AVs) and thermal imaging playing pivotal roles in achieving sustainable urban mobility. This study explores the application of the SegNet deep learning architecture for multi-class semantic segmentation of thermal images in constrained environments. The methodology encompasses data acquisition using a thermal camera in urban settings, annotation of 3,001 thermal images across 10 object classes, and rigorous model training with a high-performance system. SegNet demonstrated robust learning capabilities, achieving a training accuracy of 96.7% and a final loss of 0.096 after 120 epochs. Testing results revealed strong performance for distinct objects like motorcycles (F1 score: 0.63) and poles (F1 score: 0.84), but challenges in segmenting complex patterns such as buildings (F1 score: 0.34) and trees (F1 score: 0.42). Visual analysis corroborated these findings, highlighting strengths in segmenting well-defined objects while addressing difficulties in handling variability and elongated structures. Despite these limitations, the study establishes SegNet's potential for thermal image segmentation in AV systems. This research contributes to the advancement of computer vision in autonomous navigation, fostering sustainable and green transportation solutions while emphasizing areas for further refinement to enhance performance in complex environments.
Downloads
References
R. D. Saniyyah, U. Islam, N. Sunan, and A. Surabaya, “Peran Inovasi Teknologi Dalam Green Transportasi : Mewujudkan Green Economy Dan Pembangunan,” 2024.
M. Rauf, L. Kumar, S. A. Zulkifli, and A. Jamil, “Aspects of artificial intelligence in future electric vehicle technology for sustainable environmental impact,” Environmental Challenges, vol. 14, Jan. 2024, doi: https://doi.org/10.1016/j.envc.2024.100854.
A. Tijani, “Obstacle Avoidance Path Design for Autonomous Vehicles – A Review,” vol. 3, no. 5, pp. 64–81, 2021.
D. Garikapati, “Autonomous Vehicles : Evolution of Artificial Intelligence and the Current Industry Landscape,” 2024.
S. More, A. Singh, P. Jana, and B. Pal, “Perception and Planning in Autonomous Car,” no. December, 2020, doi: https://doi.org/10.13140/RG.2.2.31312.38401.
A. B. Pratama, R. Effendi, A. Kadir, and A. Hady, “Deteksi Ruang Kosong pada Jalan Menggunakan Semantic Segmentation pada Mobil Otonom,” vol. 11, no. 1, 2022.
A. Silwal, T. Parhar, F. Yandun, and G. Kantor, “A Robust Illumination-Invariant Camera System for Agricultural Applications,” Jan. 2021, doi: https://doi.org/10.1109/IROS51168.2021.9636542.
T. Sefer and R. Ayaz, “Performance investigation of different headlights used in vehicles under foggy conditions,” Sci Rep, pp. 1–12, 2023, doi: https://doi.org/10.1038/s41598-023-31883-3.
J. Manuel et al., “Analysis of Thermal Imaging Performance under Extreme Foggy Conditions : Applications to Autonomous Driving,” 2022.
A. N. Wilson, K. A. Gupta, B. H. Koduru, A. Kumar, A. Jha, and L. R. Cenkeramaddi, “Recent Advances in Thermal Imaging and its Applications Using Machine Learning: A Review,” IEEE Sensors Journal, vol. 23, no. 4, pp. 3395–3407, 2023, doi: https://doi.org/10.1109/JSEN.2023.3234335.
Y. Guo and B. Yang, “A Survey of Semantic Segmentation Methods in Traffic Scenarios,” in 2022 International Conference on Machine Learning, Cloud Computing and Intelligent Mining (MLCCIM), 2022, pp. 452–457. doi: https://doi.org/10.1109/MLCCIM55934.2022.00083.
X. Zhao, L. Wang, Y. Zhang, X. Han, and M. Deveci, A review of convolutional neural networks in computer vision, vol. 57, no. 4. Springer Netherlands, 2024. doi: https://doi.org/10.1007/s10462-024-10721-6.
M. Laksono, A. Satyawan, and S. Siswanti, “Segmentasi Objek Berbasis Kamera Termal Menggunakan Deep Learning (Pre-Trained Resnet 34)Thermal Image-Based Object Segmentation Using Deep Learning (Pre-Trained Resnet 34),” Prosiding Seminar Nasional Sains Teknologi dan Inovasi Indonesia (SENASTINDO), vol. 4, pp. 333–343, 2022, doi: https://doi.org/10.54706/senastindo.v4.2022.210.
R. Fauzan, A. Satyawan, S. Siswanti, and H. Puspita, “Segmentasi Objek Berbasis Gambar Termal Menggunakan Deep Learning (Pre-Trained Resnext 50),” vol. 4, no. September, pp. 308–319, 2022.
U. Ulusoy, O. Eren, and A. Demi̇rhan, “Development of an obstacle avoiding autonomous vehicle by using stereo depth estimation and artificial intelligence based semantic segmentation,” Engineering Applications of Artificial Intelligence, vol. 126, p. 106808, 2023, doi: https://doi.org/10.1016/j.engappai.2023.106808.
A. Helnawan, M. Attamimi, and A. N. Irfansyah, “Sistem Segmentasi Jalan dan Objek untuk Kendaraan Otonom Menggunakan Kamera RGB-D,” Jurnal Teknik ITS, vol. 12, no. 1, 2023, doi: https://doi.org/10.12962/j23373539.v12i1.110848.
C. Zhang, W. Lu, J. Wu, C. Ni, and H. Wang, “SegNet Network Architecture for Deep Learning Image Segmentation and Its Integrated Applications and Prospects,” vol. 9, no. 2, 2024.
E. Chamseddine, L. Tlig, M. Sayadi, and M. Bouchouicha, “SegNet Architecture for Dermoscopic Image Segmentation,” in 2022 IEEE Information Technologies & Smart Industrial Systems (ITSIS), 2022, pp. 1–6. doi: https://doi.org/10.1109/ITSIS56166.2022.10118404.
M. Yumuş, M. Apaydın, A. Değirmenci, H. Kaplanoğlu, S. Kesikburun, and Ö. Karal, “Deep Convolutional Neural Networks Using SegNet for Automatic Spinal Canal Segmentation in Axial MRI,” in 2023 Innovations in Intelligent Systems and Applications Conference (ASYU), 2023, pp. 1–6. doi: https://doi.org/10.1109/ASYU58738.2023.10296627.
S. Khrueakhrai and J. Srinonchat, “Railway Track Detection Based on SegNet Deep Learning,” in TENCON 2023 - 2023 IEEE Region 10 Conference (TENCON), 2023, pp. 409–413. doi: https://doi.org/10.1109/TENCON58879.2023.10322378.
G. Paul and G. Ramkumar, “An Automated Classification and Segmentation of Fatty Liver Disease Using SegNet Model on Ultrasound Images,” in 2023 International Conference on Emerging Research in Computational Science (ICERCS), 2023, pp. 1–6. doi: https://doi.org/10.1109/ICERCS57948.2023.10434261.
K. R. Akshatha, A. K. Karunakar, S. B. Shenoy, and A. K. Pai, “Human Detection in Aerial Thermal Images Using Faster R-CNN and SSD Algorithms,” pp. 1–15, 2022.
