Deep Learning; Lidar; Semantic Segmentation; Urban environment; 3D city models; 3D point cloud; Aerial images
Abstract :
[en] Semantic segmentation in a large-scale urban environment is crucial for a deep and rigorous understanding of urban environments. The development of Lidar tools in terms of resolution and precision offers a good opportunity to satisfy the need of developing 3D city models. In this context, deep learning revolutionizes the field of computer vision and demonstrates a good performance in semantic segmentation. To achieve this objective, we propose to design a scientific methodology involving a method of deep learning by integrating several data sources (Lidar data, aerial images, etc) to recognize objects semantically and automatically. We aim at extracting automatically the maximum amount of semantic information in a urban environment with a high accuracy and performance.
Disciplines :
Earth sciences & physical geography
Author, co-author :
Ballouch, Zouhair ; Université de Liège - ULiège > Sphères ; IAV Hassan II, College of Geomatic Sciences and Surveying Engineering, Rabat, Morocco
Hajji, R.; IAV Hassan II, College of Geomatic Sciences and Surveying Engineering, Rabat, Morocco
Ettarid, M.; IAV Hassan II, College of Geomatic Sciences and Surveying Engineering, Rabat, Morocco
Language :
English
Title :
The contribution of deep learning to the semantic segmentation of 3D point-clouds in urban areas
Publication date :
May 2020
Event name :
2020 IEEE International conference of Moroccan Geomatics (Morgeo)
Event date :
2020
Main work title :
Proceedings - 2020 IEEE International Conference of Moroccan Geomatics, MORGEO 2020, art. no. 9121898, .
Publisher :
Institute of Electrical and Electronics Engineers Inc.
Haoyi. Xiu, Poliyapram. Vinayaraj, Kyoung-Sook. Kim, Ryosuke. Nakamura, Wanglin. Yan, "3D Semantic Segmentation for High-resolution Aerial Survey Derived Point Clouds using Deep Learning (Demonstration), " Information Systems (SIGSPATIAL '18), November 6-9, 2018, Seattle, WA, USA, Farnoush Banaei-Kashani and Erik Hoel (Eds.). ACM, New York, NY, USA, 2018.
Patrik. Tosteberg, " Semantic Segmentation of Point clouds using Deep Learning, " Master of science Thesis in Electrical Engineering, Department of Electrical Engineering, Linkoping University, 2017.
Francis. Engelmann, Theodora. Kontogianni, Jonas. Schult, Bastian. Leibe, "Know What Your Neighbors Do: 3D Semantic Segmentation of Point Clouds, " RWTH Aachen University, Aachen, Germany, 2018.
Bellakaout. Abdelmounaim, "Extraction automatique des batiments, vegetation et voirie a partir des donnees Lidar 3D, " These de docteur de l'institut agronomique et veterinaire Hassan II, Maroc, 2016.
Zixiang. Zhou, Jie. Gong, "Automated residential building detection from airborne Lidar data with deep neural networks, " Advanced Engineering Informatics 36 229-241, 2018.
Capri. A, " caracterisation des objets dans une image en vue d'une aide a l'interpretation et d'une compression adaptee au contenu: application aux images echographiques, "2007.
Lee. I, Schenk. T, "Perceptual organization of 3D surface points, " Photogrammetric computer vision. ISPRS comm. Ill, Graz, Austria. Vol. XXXIV, part3A/B, ISSN 1682-1750, 2002.
Wang. M, Tseng. Y. H, "Lidar data segmentation and classification based on octree structure, " Int. International Archives of Photogrammetry and Remote Sensing, ISSN 1682-1750, Vol, XXXV, partB3, 2004.
Filin. S, Pleifer. N, " Segmentation of airborne laser scanning data using a slope adaptive neighbourhood, " ISPRS Journal of Photogrammetry and Remote Sensing 60 (2006) 71-80, 2006.
Lari. Z, A. Habib, E. Kwak, "An adaptive approach for segmentation of 3D laser point cloud, "ISPRS Workshop Laser Scanning, Calgary, Canada, 2011.
Lari. Z, A. Habib, "Segmentation-based classification of laser scanning data, "ASPRS 2012 Annual Conference, Sacramento, California, 2012.
Heath. M, S. Sarkar, T. Sanocki, K. Bowyery, " Comparison of edge detectors a methodology and initial study, "Computer Vision and Image Understanding, 69, 38-54, 1998.
Jiang. X, H. Bunke, "Edge detection in range images based on scan line approximation, " Computer Vision and Image Understanding, 73, 183199, 1999.
Sappa. A. D, M. Devy, "Fast range image segmentation by an edge detection strategy, " Proceedings of 3nd International Conference on 3-D Digital Imaging and Modeling, 292-299, 2001.
Beale. D, Iravani. P, Hall. P, "Probabilistic Models for Robot-Based Object Segmentation, " Robotics and Autonomous Systems, vol. 59, issue 12, pp. 1080-1089, 2011.
Jeonghyeon. Wang, Jinwhan. Kim, " Semantic Segmentation of Urban Scenes with a Location Prior Map Using Lidar Measurements, " IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) September 24-28, 2017, Vancouver, BC, Canada, 2017.
Mehmet. Ali. Cagn. Tuncer, Dirk. Schulz, "A Framework for the Segmentation and Classification of 3D Point Clouds using Temporal, Spatial and Semantic Information, " Cognitive Mobile Systems, Fraunhofer FKIE, Fraunhoferstr. 20, 53343 Wachtberg, Germany.
Mohammad. Awrangjeb, Mehdi. Ravanbakhsh, Clive S. Fraser, "Automatic detection of residential buildings using LIDAR data and multispectral imagery, " ISPRS Journal of Photogrammetry and Remote Sensing 65 (2010) 457-467, 2010.
S. A. N. Gilani, M. Awrangjebb, G. Lu, " FUSION OF LIDAR DATA AND MULTISPECTRAL IMAGERY FOR EFFECTIVE BUILDING DETECTION BASED ON GRAPH AND CONNECTED COMPONENT ANALYSIS, " The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-3/W2, 2015 PIA15+HRIGI15-Joint ISPRS conference 2015, 25-27 March 2015, Munich, Germany, 2015.
Ying. Sun, Xinchang. Zhang, Qinchuan. Xin, Jianfeng. Huang, "Developing a multi-filter convolutional neural network for semantic segmentation using high-resolution aerial imagery and Lidar data, " ISPRS Journal of Photogrammetry and Remote Sensing, 2018.
Rui. Zhanga, Guangyun. Lia, Minglei. Lia, Li. Wanga, " Fusion of images and point clouds for the semantic segmentation of largescale 3D scenes based on deep learning, "ISPRS Journal of Photogrammetry and Remote Sensing, 2018.
Everingham. M, Van. Gool. L, Williams. C. K. I, Winn. J, Zisserman. A, Int. J. Comp. Vis. Ill (1), 98-136, 2015.
David. Griffiths, Jan. Boehm, " Improving public data for building segmentation from Convolutional Neural Networks (CNNs) for fused airborne Lidar and image data using active contours, "ISPRS Journal of Photogrammetry and Remote Sensing 154 (2019) 70-83, 2019.
Li. Y, "Deep learning for remote sensing image classification: A survey. Wiley Interdisciplinary Reviews, " Data Mining and Knowledge Discovery. Vol. 8, pp. 1264, 2018.
Garcia-Garcia. A, "A Review on Deep Learning Techniques Applied to Semantic Segmentation, " ArXiv press. 23 p, 2017.
S. OUATTARA, A. CLEMENT, "Etiquetage d'histogrammes multidimensionnels compacts pour l'analyse d'images multicomposantes, " Colloque GRETSI, Troyes, 11-14 septembre 2007.
Zisserman. A, Simonyan. K, "Very Deep Convolutional Networks for Large-Scale Image Recognition, " arXiv print. 14p, 2014.
A. Boulch, J. Guerry, B. L. Saux, and N. Audebert, "Snapnet: 3d point cloud semantic labeling with 2d deep segmentation networks, " Computers Graphics, vol. 71, pp. 189-198, 2018.
L. P. Tchapmi, C. B. Choy, I. Armeni, J. Gwak, and S. Savarese, "Segcloud: Semantic segmentation of 3d point clouds, " 2017 International Conference on 3D Vision (3DV), pp. 537-547, 2017.
Vijay. B, Kendall. A, Cipolla. R, "SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation, " IEEE transactions on pattern analysis and machine intelligence. Vol. 39, pp. 2481-2495, 2017.
Bisheng. Yang, Zhen. Dong, Yuan. Liu, Fuxun. Liang, Yongjun. Wang, " Computing multiple aggregation levels and contextual features for road facilities recognition using mobile laser scanning data, " ISPRS Journal of Photogrammetry and Remote Sensing, Volume 126, pp. 180-194, 2017.
C. R. Qi, H. Su, K. Mo, and L. J. Guibas, "Pointnet: Deep learning on point sets for 3d classification and segmentation, " CoRR, vol. abs/1612. 00593, 2016.
Loic. Landrieu, Martin. Simonovsky, "Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs, " The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4558-4567, 2018.
Loic. Landrieu, "Segmentation semantique de grands nuages de points avec Superpoint Graph, " MATIS-IGN, 2018.
Nicolas. Audebert, Alexandre. Boulch, Bertrand. Le Saux, Sebastien. Lefevre, "Segmentation semantique profonde par regression sur cartes de distances signees, " Reconnaissance des Formes, Image, Apprentissage et Perception (RFIAP), Marne-la-Vallee, France. hal-01809991, Jun 2018.
Chakri. Lowphansirikul, Kyoung-Sook. Kim, Poliyapram. Vinayaraj, and Suppawong. Tuarob, "3D Semantic Segmentation of Large-Scale Point-Clouds in Urban Areas Using Deep Learning, " International Conference on Knowledge and Smart Technology (KST), 2019. DOI: 10. 1109/KST. 2019. 8687813
T. Hackel, N. Savinov, L. Ladicky, J. D. Wegner, K. Schindler, andM. Pollefeys, "SEMANTIC3D. NET: A new large-scale point cloud classification benchmark, " in ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. IV-l-Wl, pp. 91-98, 2017.
