This study is part of the first author’s Ph.D. thesis. We thank the dataset providers. The original data used in this study are publicly available at: https://ressources.data.sncf.com/explore/dataset/nuage-points-3d for French data, and at https://data.mendeley.com/datasets/ccxpzhx9dj for Hungarian data (accessed on 28 November 2023).
[en] Railway scene understanding is crucial for various applications, including autonomous trains, digital twining, and infrastructure change monitoring. However, the development of the latter is constrained by the lack of annotated datasets and limitations of existing algorithms. To address this challenge, we present Rail3D, the first comprehensive dataset for semantic segmentation in railway environments with a comparative analysis. Rail3D encompasses three distinct railway contexts from Hungary, France, and Belgium, capturing a wide range of railway assets and conditions. With over 288 million annotated points, Rail3D surpasses existing datasets in size and diversity, enabling the training of generalizable machine learning models. We conducted a generic classification with nine universal classes (Ground, Vegetation, Rail, Poles, Wires, Signals, Fence, Installation, and Building) and evaluated the performance of three state-of-the-art models: KPConv (Kernel Point Convolution), LightGBM, and Random Forest. The best performing model, a fine-tuned KPConv, achieved a mean Intersection over Union (mIoU) of 86%. While the LightGBM-based method achieved a mIoU of 71%, outperforming Random Forest. This study will benefit infrastructure experts and railway researchers by providing a comprehensive dataset and benchmarks for 3D semantic segmentation. The data and code are publicly available for France and Hungary, with continuous updates based on user feedback.
Ballouch, Zouhair ; Université de Liège - ULiège > Sphères ; College of Geomatic Sciences and Surveying Engineering, Hassan II Institute of Agronomy and Veterinary Medicine, Rabat 10101, Morocco
Hajji, Rafika ; Université de Liège - ULiège > Département de géographie > Geospatial Data Science and City Information Modelling (GeoScITY) ; College of Geomatic Sciences and Surveying Engineering, Hassan II Institute of Agronomy and Veterinary Medicine, Rabat 10101, Morocco
This study is part of the first author’s Ph.D. thesis. We thank the dataset providers. The original data used in this study are publicly available at: https://ressources.data.sncf.com/explore/dataset/nuage-points-3d for French data, and at https://data.mendeley.com/datasets/ccxpzhx9dj for Hungarian data (accessed on 28 November 2023).
scite shows how a scientific paper has been cited by providing the context of the citation, a classification describing whether it supports, mentions, or contrasts the cited claim, and a label indicating in which section the citation was made.
Bibliography
Soilán M. Sánchez-Rodríguez A. Del Río-Barral P. Perez-Collazo C. Arias P. Riveiro B. Review of laser scanning technologies and their applications for road and railway infrastructure monitoring Infrastructures 2019 4 58 10.3390/infrastructures4040058
Lamas D. Soilán M. Grandío J. Riveiro B. Automatic point cloud semantic segmentation of complex railway environments Remote Sens. 2021 13 2332 10.3390/rs13122332
Chen X. Chen Z. Liu G. Chen K. Wang L. Xiang W. Zhang R. Railway overhead contact system point cloud classification Sensors 2021 21 4961 10.3390/s21154961
Roynard X. Deschaud J.-E. Goulette F. Paris-Lille-3D: A Point Cloud Dataset for Urban Scene Segmentation and Classification Available online: http://caor-mines-paristech.fr/fr/ (accessed on 14 December 2023)
Tan W. Qin N. Ma L. Li Y. Du J. Cai G. Yang K. Li J. Toronto-3D: A Large-scale Mobile LiDAR Dataset for Semantic Segmentation of Urban Roadways Available online: https://www.cloudcompare.org (accessed on 14 December 2023)
Behley J. Garbade M. Milioto A. Quenzel J. Behnke S. Stachniss C. Gall J. SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences April 2019 Available online: http://arxiv.org/abs/1904.01416 (accessed on 14 December 2023)
Lytkin S. Badenko V. Fedotov A. Vinogradov K. Chervak A. Milanov Y. Zotov D. Saint Petersburg 3D: Creating a Large-Scale Hybrid Mobile LiDAR Point Cloud Dataset for Geospatial Applications Remote Sens. 2023 15 2735 10.3390/rs15112735
Roynard X. Deschaud J.-E. Goulette F. Paris-Lille-3D: A Large and high-quality ground truth urban point cloud dataset for automatic segmentation and classification arXiv 2017 1712.00032 Available online: http://arxiv.org/abs/1712.00032 (accessed on 14 December 2023) 10.1177/0278364918767506
Zhang R. Wu Y. Jin W. Meng X. Deep-Learning-Based Point Cloud Semantic Segmentation: A Survey Electronics 2023 12 3642 10.3390/electronics12173642
Oh K. Yoo M. Jin N. Ko J. Seo J. Joo H. Ko M. A Review of Deep Learning Applications for Railway Safety Appl. Sci. 2022 12 572 10.3390/app122010572
He Y. Yu H. Liu X. Yang Z. Sun W. Mian A. Deep Learning Based 3D Segmentation: A Survey arXiv 2021 2103.05423 Available online: http://arxiv.org/abs/2103.05423 (accessed on 14 December 2023)
Schnabel R. Wahl R. Klein R. Efficient RANSAC for Point-Cloud Shape Detection Comput. Graph. Forum 2007 26 214 226 10.1111/j.1467-8659.2007.01016.x
Truong Q.H. Knowledge-Based 3D Point Clouds Processing 2014 Available online: https://theses.hal.science/tel-00977434 (accessed on 14 December 2023)
Ponciano J.-J. Roetner M. Reiterer A. Boochs F. Object Semantic Segmentation in Point Clouds—Comparison of a Deep Learning and a Knowledge-Based Method ISPRS Int. J. Geo-Inf. 2021 10 256 10.3390/ijgi10040256
Alkadri M.F. Alam S. Santosa H. Yudono A. Beselly S.M. Investigating Surface Fractures and Materials Behavior of Cultural Heritage Buildings Based on the Attribute Information of Point Clouds Stored in the TLS Dataset Remote Sens. 2022 14 410 10.3390/rs14020410
Su H. Maji S. Kalogerakis E. Learned-Miller E. Multi-View Convolutional Neural Networks for 3D Shape Recognition Available online: http://vis-www.cs.umass.edu/mvcnn (accessed on 14 December 2023)
Hamdi A. Giancola S. Ghanem B. MVTN: Multi-View Transformation Network for 3D Shape Recognition Available online: https://github.com/ajhamdi/MVTN (accessed on 14 December 2023)
Dai A. Nießner M. 3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation arXiv 2018 1803.10409 Available online: http://arxiv.org/abs/1803.10409 (accessed on 14 December 2023)
Kundu A. Yin X. Fathi A. Ross D. Brewington B. Funkhouser T. Pantofaru C. Virtual Multi-view Fusion for 3D Semantic Segmentation arXiv 2020 2007.13138 Available online: http://arxiv.org/abs/2007.13138 (accessed on 14 December 2023)
Boulch A. Guerry J. Le Saux B. Audebert N. SnapNet: 3D point cloud semantic labeling with 2D deep segmentation networks Comput. Graph. 2018 71 189 198 10.1016/j.cag.2017.11.010
Yang Y. Wu X. He T. Zhao H. Liu X. SAM3D: Segment Anything in 3D Scenes arXiv 2023 2306.03908 Available online: http://arxiv.org/abs/2306.03908 (accessed on 14 December 2023)
Wu B. Wan A. Yue X. Keutzer K. SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloud arXiv 2017 1710.07368 Available online: http://arxiv.org/abs/1710.07368 (accessed on 14 December 2023)
Wang Y. Shi T. Yun P. Tai L. Liu M. PointSeg: Real-Time Semantic Segmentation Based on 3D LiDAR Point Cloud arXiv 2018 1807.06288 Available online: http://arxiv.org/abs/1807.06288 (accessed on 14 December 2023)
Karara G. Hajji R. Poux F. 3D point cloud semantic augmentation: Instance segmentation of 360° panoramas by deep learning techniques Remote Sens. 2021 13 3647 10.3390/rs13183647
Ando A. Gidaris S. Bursuc A. Puy G. Boulch A. Marlet R. Marlet. RangeViT: Towards Vision Transformers for 3D Semantic Segmentation in Autonomous Driving arXiv 2023 2301.10222 Available online: http://arxiv.org/abs/2301.10222 (accessed on 14 December 2023)
Xu Y. Tong X. Stilla U. Voxel-based representation of 3D point clouds: Methods, applications, and its potential use in the construction industry Autom. Constr. 2021 126 103675 10.1016/j.autcon.2021.103675
Fang Z. Xiong B. Liu F. Sparse point-voxel aggregation network for efficient point cloud semantic segmentation IET Comput. Vis. 2022 16 644 654 10.1049/cvi2.12131
Ye M. Wan R. Xu S. Cao T. Chen Q. DRINet++: Efficient Voxel-as-point Point Cloud Segmentation arXiv 2021 2111.08318
Li H. Guan H. Ma L. Lei X. Yu Y. Wang H. Delavar M.R. Li J. MVPNet: A multi-scale voxel-point adaptive fusion network for point cloud semantic segmentation in urban scenes Int. J. Appl. Earth Obs. Geoinf. 2023 122 103391 10.1016/j.jag.2023.103391
Hang S. Jampani V. Sun D. Maji S. Kalogerakis E. Yang M.-H. Kau J. SPLATNet: Sparse Lattice Networks for Point Cloud Processing arXiv 2018 1802.08275 Available online: http://arxiv.org/abs/1802.08275 (accessed on 14 December 2023)
Rosu R.A. Schütt P. Quenzel J. Behnke S. LatticeNet: Fast Point Cloud Segmentation Using Permutohedral Lattices arXiv 2019 1912.05905 Available online: http://arxiv.org/abs/1912.05905 (accessed on 14 December 2023)
Rosu R.A. Schütt P. Quenzel J. Behnke S. LatticeNet: Fast Spatio-Temporal Point Cloud Segmentation Using Permutohedral Lattices arXiv 2021 10.1007/s10514-021-09998-1 Available online: http://arxiv.org/abs/2108.03917 (accessed on 14 December 2023) 2108.03917
Qi C.R. Su H. Mo K. Guibas L.J. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation arXiv 2016 1612.00593 Available online: http://arxiv.org/abs/1612.00593 (accessed on 14 December 2023)
Qi C.R. Yi L. Su H. Guibas L.J. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space arXiv 2017 1706.02413 Available online: http://arxiv.org/abs/1706.02413 (accessed on 14 December 2023)
Wu W. Qi Z. Fuxin L. PointConv: Deep Convolutional Networks on 3D Point Clouds arXiv 2018 1811.07246 Available online: http://arxiv.org/abs/1811.07246 (accessed on 14 December 2023)
Thomas H. Qi C.R. Deschaud J.-E. Marcotegui B. Goulette F. Guibas L.J. KPConv: Flexible and Deformable Convolution for Point Clouds arXiv 2019 1904.08889
Zeng Z. Xu Y. Xie Z. Wan J. Wu W. Dai W. RG-GCN: A Random Graph Based on Graph Convolution Network for Point Cloud Semantic Segmentation Remote Sens. 2022 14 4055 10.3390/rs14164055
Jiang T. Sun J. Liu S. Zhang X. Wu Q. Wang Y. Hierarchical semantic segmentation of urban scene point clouds via group proposal and graph attention network Int. J. Appl. Earth Obs. Geoinf. 2021 105 102626 10.1016/j.jag.2021.102626
Landrieu L. Simonovsky M. Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs arXiv 2017 1711.09869 Available online: http://arxiv.org/abs/1711.09869 (accessed on 14 December 2023)
Li G. Müller M. Thabet A. Ghanem B. DeepGCNs: Can GCNs Go as Deep as CNNs? arXiv 2019 1904.03751 Available online: http://arxiv.org/abs/1904.03751 (accessed on 14 December 2023)
Lu D. Xie Q. Wei M. Gao K. Xu L. Li J. Transformers in 3D Point Clouds: A Survey arXiv 2022 2205.07417 Available online: http://arxiv.org/abs/2205.07417 (accessed on 14 December 2023)
Zhao H. Jiang L. Jia J. Torr P. Koltun V. Point Transformer arXiv 2020 2012.09164 Available online: http://arxiv.org/abs/2012.09164 (accessed on 14 December 2023)
Lai X. Liu J. Jiang L. Wang L. Zhao H. Liu S. Qi X. Jia J. Stratified Transformer for 3D Point Cloud Segmentation arXiv 2022 2203.14508 Available online: http://arxiv.org/abs/2203.14508 (accessed on 14 December 2023)
Zhou J. Xiong Y. Chiu C. Liu F. Gong X. SAT: Size-Aware Transformer for 3D Point Cloud Semantic Segmentation arXiv 2023 2301.06869 Available online: http://arxiv.org/abs/2301.06869 (accessed on 14 December 2023)
Fei B. Yang W. Liu L. Luo T. Zhang R. Li Y. He Y. Self-supervised Learning for Pre-Training 3D Point Clouds: A Survey arXiv 2023 2305.04691 Available online: http://arxiv.org/abs/2305.04691 (accessed on 14 December 2023)
Lin X. Luo H. Guo W. Wang C. Li J. A Multi-task Learning Framework for Semantic Segmentation in MLS Point Clouds Artificial Intelligence and Security Lecture Notes in Computer Science (Including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) Springer Science and Business Media Deutschland GmbH Berlin/Heidelberg, Germany 2022 382 392 10.1007/978-3-031-06794-5_31
Kirillov A. Mintun E. Ravi N. Mao H. Rolland C. Gustafson L. Xiao T. Whitehead S. Berg A.C. Lo W.-Y. et al. Segment Anything arXiv 2023 2304.02643 Available online: http://arxiv.org/abs/2304.02643 (accessed on 14 December 2023)
Hong Y. Zhen H. Chen P. Zheng S. Du Y. Chen Z. Gan C. 3D-LLM: Injecting the 3D World into Large Language Models arXiv 2023 2307.12981 Available online: http://arxiv.org/abs/2307.12981 (accessed on 14 December 2023)
Arastounia M. Automated recognition of railroad infrastructure in rural areas from LIDAR data Remote Sens. 2015 7 14916 14938 10.3390/rs71114916
Vosselman G. Klein R. Visualisation and structuring of point clouds Airborne and Terrestrial Laser Scanning Vosselman M.G. Maas H.G. CRC Press (Taylor & Francis) Boca Raton, FL, USA 2010 45 81
Muja M. Lowe D.G. Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration VISAPP 2009, Proceedings of the Fourth International Conference on Computer Vision Theory and Applications, Lisboa, Portugal, 5–8 February 2009 Ranchordas A. Araújo H. INSTICC Press Lisbon, Portugal 2009 Volume 1 331 340
Chen L. Jung J. Sohn G. Multi-Scale Hierarchical CRF for Railway Electrification Asset Classification from Mobile Laser Scanning Data IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2019 12 3131 3148 10.1109/JSTARS.2019.2918272
Grandio J. Riveiro B. Soilán M. Arias P. Point cloud semantic segmentation of complex railway environments using deep learning Autom. Constr. 2022 141 104425 10.1016/j.autcon.2022.104425
Ton B. Ahmed F. Linssen J. Semantic Segmentation of Terrestrial Laser Scans of Railway Catenary Arches: A Use Case Perspective Sensors 2023 23 222 10.3390/s23010222
Zendel O. Murschitz M. Zeilinger M. Steininger D. Abbasi S. Beleznai C. RailSem19: A Dataset for Semantic Rail Scene Understanding Available online: www.wilddash.cc (accessed on 14 December 2023)
Harb J. Rébéna N. Chosidow R. Roblin G. Potarusov R. Hajri H. FRSign: A Large-Scale Traffic Light Dataset for Autonomous Trains arXiv 2020 2002.05665 Available online: http://arxiv.org/abs/2002.05665 (accessed on 14 December 2023)
Toprak T. Aydın B. Belenlioğlu B. Güzeliş C. Selver M.A. Railway Pedestrian Dataset (RAWPED) Zenodo 2020 Available online: https://zenodo.org/records/3741742 (accessed on 14 December 2023)
Toprak T. Belenlioglu B. Aydin B. Guzelis C. Selver M.A. Conditional Weighted Ensemble of Transferred Models for Camera Based Onboard Pedestrian Detection in Railway Driver Support Systems IEEE Trans. Veh. Technol. 2020 69 5041 5054 10.1109/TVT.2020.2983825
Leibner P. Hampel F. Schindler C. GERALD: A novel dataset for the detection of German mainline railway signals Proc. Inst. Mech. Eng. Part F J. Rail Rapid Transit. 2023 237 1332 1342 10.1177/09544097231166472
Tagiew R. Klasek P. Tilly R. Köppel M. Denzler P. Neumaier P. Klockau T. Boekhoff M. Schwalbe K. OSDaR23: Open Sensor Data for Rail 2023 arXiv 2023 2305.03001 Available online: http://arxiv.org/abs/2305.03001 (accessed on 14 December 2023)
WHU-Railway3D: A Diverse Dataset and Benchmark for Railway Point Cloud Semantic Segmentation Available online: https://github.com/WHU-USI3DV/WHU-Railway3D (accessed on 13 March 2024)
Ton B. Labelled High Resolution Point Cloud Dataset of 15 Catenary Arches in The Netherlands 4TU.ResearchData Delft, The Netherlands 2022 10.4121/17048816.v1
Tagiew R. Köppel M. Schwalbe K. Denzler P. Neumaier P. Klockau T. Boekhoff M. Klasek P. Tilly R. Open Sensor Data for Rail 2023 arXiv 2023 2305.03001 10.57806/9MV146R0
Eastepp M. Faris L. Ricks K. UA_L-DoTT: University of Alabama’s large dataset of trains and trucks Data Brief 2022 42 108073 10.1016/j.dib.2022.108073
D’Amico G. Marinoni M. Nesti F. Rossolini G. Buttazzo G. Sabina S. Lauro G. TrainSim: A Railway Simulation Framework for LiDAR and Camera Dataset Generation arXiv 2023 2302.14486 Available online: http://arxiv.org/abs/2302.14486 (accessed on 14 December 2023) 10.1109/TITS.2023.3297728
Fayjie R. Vandewalle P. Few-shot learning on point clouds for railroad segmentation Electron. Imaging 2023 35 100-1 100-5 10.2352/EI.2023.35.17.3DIA-100
Wang Y. Railway SLAM Dataset IEEE Dataport Piscataway, NJ, USA 2022 10.21227/kq9s-ct48
Corongiu M. Masiero A. Tucci G. Classification of railway assets in mobile mapping point clouds International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences—ISPRS Archives International Society for Photogrammetry and Remote Sensing Hanover, Germany 2020 219 225 10.5194/isprs-archives-XLIII-B1-2020-219-2020
Riquelme J.L.A. Ruffo M. Tomás R. Riquelme A. Pagán J.I. Cano M. Pastor J.L. 3D Point Cloud of a Railway Slope—MOMIT (Multi-Scale Observation and Monitoring of Railway Infrastructure Threats) EU Project—H2020-EU.3.4.8.3.—Grant Agreement ID: 777630 Zenodo 2020 Available online: https://zenodo.org/records/3777996 (accessed on 14 December 2023) 10.5281/zenodo.3777996
Cserep M. Hungarian MLS Point Clouds of Railroad Environment and Annotated Ground Truth Data Mendeley Data 2022 Available online: https://data.mendeley.com/datasets/ccxpzhx9dj/1 (accessed on 28 November 2023) 10.17632/ccxpzhx9dj.1
González-Collazo S.M. Balado J. González E. Nurunnabi A. A discordance analysis in manual labelling of urban mobile laser scanning data used for deep learning based semantic segmentation Expert Syst. Appl. 2023 230 120672 10.1016/j.eswa.2023.120672
Girardeau-Montaut G. CloudCompare 12 July 2023 Available online: https://www.cloudcompare.org/ (accessed on 14 December 2023)
De Gélis I. Lefèvre S. Corpetti T. Siamese KPConv: 3D multiple change detection from raw point clouds using deep learning ISPRS J. Photogramm. Remote Sens. 2023 197 274 291 10.1016/j.isprsjprs.2023.02.001
Sevgen E. Abdikan S. Classification of Large-Scale Mobile Laser Scanning Data in Urban Area with LightGBM Remote Sens. 2023 15 3787 10.3390/rs15153787
Letard M. Lague D. Le Guennec A. Lefèvre S. Feldmann B. Leroy P. Girardeau-Montaut D. Corpetti T. 3DMASC: Accessible, explainable 3D point clouds classification. Application to Bi-Spectral Topo-Bathymetric lidar data ISPRS J. Photogramm. Remote Sens. 2024 207 175 197 Available online: https://hal.science/hal-04072068 (accessed on 14 December 2023) 10.1016/j.isprsjprs.2023.11.022
Li Y. Fan C. Wang X. Duan Y. SPNet: Multi-Shell Kernel Convolution for Point Cloud Semantic Segmentation arXiv 2021 2109.11610 Available online: http://arxiv.org/abs/2109.11610 (accessed on 14 December 2023)
Thomas H. Qi C.R. Deschaud J.-E. Marcotegui B. Goulette F. Guibas L.J. KPConv: Flexible and Deformable Convolution for Point Clouds Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Seoul, Republic of Korea 24 October–2 November 2019
Thomas H. Deschaud J.-E. Marcotegui B. Goulette F. Le Gall Y. Semantic Classification of 3D Point Clouds with Multiscale Spherical Neighborhoods arXiv 2018 1808.00495 Available online: http://arxiv.org/abs/1808.00495 (accessed on 14 December 2023)
Ke G. Meng Q. Finley T. Wang T. Chen W. Ma W. Ye Q. Liu T.-Y. LightGBM: A Highly Efficient Gradient Boosting Decision Tree 2017 Available online: https://github.com/Microsoft/LightGBM (accessed on 14 December 2023)
Grandini M. Bagli E. Visani G. Metrics for Multi-Class Classification: An Overview arXiv 2020 2008.05756 Available online: http://arxiv.org/abs/2008.05756 (accessed on 14 December 2023)
González E. Balado J. Arias P. Lorenzo H. Realistic correction of sky-coloured points in Mobile Laser Scanning point clouds Opt. Laser Technol. 2022 149 107807 10.1016/j.optlastec.2021.107807
Similar publications
Sorry the service is unavailable at the moment. Please try again later.
This website uses cookies to improve user experience. Read more
Save & Close
Accept all
Decline all
Show detailsHide details
Cookie declaration
About cookies
Strictly necessary
Performance
Strictly necessary cookies allow core website functionality such as user login and account management. The website cannot be used properly without strictly necessary cookies.
This cookie is used by Cookie-Script.com service to remember visitor cookie consent preferences. It is necessary for Cookie-Script.com cookie banner to work properly.
Performance cookies are used to see how visitors use the website, eg. analytics cookies. Those cookies cannot be used to directly identify a certain visitor.
Used to store the attribution information, the referrer initially used to visit the website
Cookies are small text files that are placed on your computer by websites that you visit. Websites use cookies to help users navigate efficiently and perform certain functions. Cookies that are required for the website to operate properly are allowed to be set without your permission. All other cookies need to be approved before they can be set in the browser.
You can change your consent to cookie usage at any time on our Privacy Policy page.
