Deep learning; Lidar; Semantic segmentation; Urban environment
Abstract :
[en] Semantic segmentation of Lidar data using Deep Learning (DL) is a fundamental step for a deep and rigorous understanding of large-scale urban areas. Indeed, the increasing development of Lidar technology in terms of accuracy and spatial resolution offers a best opportunity for delivering a reliable semantic segmentation in large-scale urban environments. Significant progress has been reported in this direction. However, the literature lacks a deep comparison of the existing methods and algorithms in terms of strengths and weakness. The aim of the present paper is therefore to propose an objective review about these methods by highlighting their strengths and limitations. We then propose a new approach based on the combination of Lidar data and other sources in conjunction with a Deep Learning technique whose objective is to automatically extract semantic information from airborne Lidar point clouds by enhancing both accuracy and semantic precision compared to the existing methods. We finally present the first results of our approach.
Disciplines :
Earth sciences & physical geography
Author, co-author :
Ballouch, Zouhair ; Université de Liège - ULiège > Sphères ; College of Geomatic Sciences and Surveying Engineering, IAV Hassan II, Rabat, Morocco
Hajji, Rafika; College of Geomatic Sciences and Surveying Engineering, IAV Hassan II, Rabat, Morocco
Ettarid, Mohamed; College of Geomatic Sciences and Surveying Engineering, IAV Hassan II, Rabat, Morocco
Language :
English
Title :
Toward a Deep Learning Approach for Automatic Semantic Segmentation of 3D Lidar Point Clouds in Urban Areas
A. Bellakaout, Extraction automatique des batiments, végétation et voirie à partir des données Lidar 3D. Thèse de docteur de l’institut agronomique et vétérinaire Hassan II, Maroc (2016)
L. Haifeng, Unsupervised scene adaptation for semantic segmentation of urban mobile laser scanning point clouds. ISPRS J. Photogramm. Remote. Sens. 169, 253–267 (2020)
B. Kim, Highway driving dataset for semantic video segmentation. School of Electrical Engineering Korea Advanced Institute of Science and Technology (KAIST), South Korea (2016)
J. Castillo-Navarro, Réseaux de neurones semi-supervisés pour la segmentation sémantique en télédétection. Colloque GRETSI sur le Traitement du Signal et des Images, Lille, France. hal-02343961 (2019)
A. Garcia-Garcia, A review on deep learning techniques applied to semantic segmentation. arXiv:1704.06857v1 [cs.CV] (2017)
M. Awrangjeb, Automatic detection of residential buildings using LIDAR data and multispectral imagery. ISPRS J. Photogram. Remote Sens. 65, 457–467 (2010)
J. Ravaglia, Segmentation de nuages de points par octrees et analyse en composantes principales. GTMG 2014, Mar 2014, Lyon, France. hal-01376473 (2014)
I. Lee, Perceptual organization of 3D surface points, photogrammetric computer vision. ISPRS Comm. III. Graz, Austria. XXXIV part 3A/B. ISSN 1682-1750 (2002)
S. Filin, Segmentation of airborne laser scanning data using a slope adaptive neighborhood. ISPRS J. Photogramm. Remote Sens. 60, 71–80 (2006). https://doi.org/10.1016/j.isprsjprs.2005.10.005 (2005)
Z. Lari, An adaptive approach for segmentation of 3D laser point cloud, in ISPRS Workshop Laser Scanning, Calgary, Canada (2011)
Z. Lari, A. Habib, Segmentation-based classification of laser scanning data, in ASPRS 2012 Annual Conference Sacramento, California, 19–23 Mar 2012
W. Yuan, PointSeg: real-time semantic segmentation based on 3D LiDAR point cloud. arXiv:1807.06288v8 [cs.CV] (2018)
F.N. Iandola, Squeezenet: Alexnet-level accuracy with 50x fewer parameters and <1mb model size. CoRR abs/1602.07360 (2016)
B. Wu, Squeezeseg: convolutional neural nets with recurrent CRF for real-time road-object segmentation from 3d lidar point cloud. CoRR abs/1710.07368 (2017)
A. Milioto, RangeNet++: fast and accurate LiDAR semantic segmentation. German Research Foundation under Germany’s Excellence Strategy, EXC-2070-390732324 (PhenoRob) as well as grant number BE 5996/1–1, and by NVIDIA Corporation (2019)
A. Boulch, SnapNet: 3D point cloud semantic labeling with 2D deep segmentation networks. Comput. Graph. (2017)
Y. Xu, Voxel-and graph-based point cloud segmentation of 3d scenes using perceptual grouping laws. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. IV-1/W1 (2017)
G. Riegler, A. Osman Ulusoy, Octnet: learning deep 3d representations at high resolutions, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 3 (2017)
H. Riemenschneider, A. Bódis-Szomorú, Learning where to classify in multi-view semantic segmentation, in Proceedings of the European Conference on Computer Vision (ECCV) (2014)
Y. Liu, Comparison of 2D image models in segmentation performance for 3D laser point clouds. Neurocomputing (2017)
M. Awrangjeb, Automatic detection of residential buildings using LIDAR data and multispectral imagery. ISPRS J. Photogramm. Remote Sens. 65, 457–467 (2010)
Y. Sun, Developing a multi-filter convolutional neural network for semantic segmentation using high-resolution aerial imagery and LiDAR data. ISPRS J. Photogramm. Remote Sens. (2018)
H. Xiu, 3D semantic segmentation for high-resolution aerial survey derived point clouds using deep learning (Demonstration), in Information Systems (SIGSPATIAL’18), 6–9 Nov 2018, Seattle, WA, USA, ed. by F. Banaei-Kashani, E. Hoel (ACM, New York, NY, USA, 2018)
R. Zhanga, Fusion of images and point clouds for the semantic segmentation of large scale 3D scenes based on deep learning. ISPRS J. Photogramm. Remote Sens. (2018)
H. Su, V. Jampani, Splatnet: sparse lattice networks for point cloud processing, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018), pp. 2530–2539
H.-Y. Chiang, A unified point-based framework for 3D segmentation, in International Conference on 3D Vision (3DV) (2019)
A. Dai, Scannet: Richly-annotated 3d reconstructions of indoor scenes, in Proceedings of CVPR 2017 (2017)
J. Li, Building and optimization of 3D semantic map based on Lidar and camera fusion. Neurocomputing
Y. Li, Deep learning for remote sensing image classification: a survey. Wiley Interdisciplinary Reviews. Data Mining and Knowledge Discovery, vol. 8, p. 1264 (2018)
C.R. Qi, Pointnet: deep learning on point sets for 3d classification and segmentation. CoRR abs/1612.00593 (2016)
C.R. Qi, PointNet++: deep hierarchical feature learning on point sets in a metric space. arXiv:1706.02413v1 [cs.CV] (2017)
L.P. Tchapmi, Segcloud: semantic segmentation of 3d point clouds, in International Conference on 3D Vision (3DV) (2017), pp. 537–547
B. Vijay, SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39, 2481–2495 (2017)
L. Landrieu, M. Simonovsky, Large-scale point cloud semantic segmentation with superpoint graphs, in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018), pp. 4558–4567
C.R. Qi, Deep Hough voting for 3D object detection in point clouds. arXiv:1904.09664v2 [cs.CV] (2019)
B. Yang, Computing multiple aggregation levels and contextual features for road facilities recognition using mobile laser scanning data. ISPRS J. Photogramm. Remote Sens. 126, 180–194 (2017)
Q. Hu, RandLA-Net: efficient semantic segmentation of large-scale point clouds. arXiv:1911.11236v3 [cs.CV] (2020)
Z. Yang, Std: sparse-to-dense 3d object detector for point cloud, in The IEEE International Conference on Computer Vision (ICCV) (2019)
Y. Cui, Deep learning for image and point cloud fusion in autonomous driving: a review. arXiv:2004.05224v2 [cs.CV] (2020)
A. Zisserman, Very deep convolutional networks for large-scale image recognition. arXiv print. 14 p (2014)
O. Ronneberger, P. Fischer, U-Net: convolutional networks for biomedical biomedical image segmentation, in International Conference on Medical Image Computing and Computer-Assisted Intervention (2015), pp. 234–241
K. He, Deep residual learning for image recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 770–778
A. Chaurasia, Linknet: exploiting encoder representations for efficient semantic segmentation. arXiv preprint 1707.03718 (2017)
M.H. Wu, ECNet: efficient convolutional networks for side scan sonar image segmentation. Sensors 19(9), 2019 (2009). https://doi. org/10.3390/s19092009
