African mammals; CNNs; deep learning; multispecies; UAV; wildlife monitoring
Abstract :
[en] Survey and monitoring of wildlife populations are among the key elements in nature conservation. The use of unmanned aerial vehicles and light aircrafts as aerial image acquisition systems is growing, as they are cheaper alternatives to traditional census methods. However, the manual localization and identification of species within imagery can be time-consuming and complex. Object detection algorithms, based on convolutional neural networks (CNNs), have shown a good capacity for animal detection. Nevertheless, most of the work has focused on binary detection cases (animal vs. background). The main objective of this study is to compare three recent detection algorithms to detect and identify African mammal species based on high-resolution aerial images. We evaluated the performance of three multi-class CNN algorithms: Faster-RCNN, Libra-RCNN and RetinaNet. Six species were targeted: topis (Damaliscus lunatus jimela), buffalos (Syncerus caffer), elephants (Loxodonta africana), kobs (Kobus kob), warthogs (Phacochoerus africanus) and waterbucks (Kobus ellipsiprymnus). The best model was then applied to a case study using an independent dataset. The best model was the Libra-RCNN, with the best mean average precision (0.80 0.02), the lowest degree of interspecies confusion (3.5 1.4%) and the lowest false positive per true positive ratio (1.7 0.2) on the test set. This model was able to detect and correctly identify 73% of all individuals (1115), find 43 individuals of species other than those targeted and detect 84 missed individuals on our independent UAV dataset, with an average processing speed of 12 s/image. This model showed better detection performance than previous studies dealing with similar habitats. It was able to differentiate six animal species in nadir aerial images. Although limitations were observed with warthog identification and individual detection in herds, this model can save time and can perform precise surveys in open savanna.
Disciplines :
Life sciences: Multidisciplinary, general & others Agriculture & agronomy Engineering, computing & technology: Multidisciplinary, general & others
Author, co-author :
Delplanque, Alexandre ; Université de Liège - ULiège > Département GxABT > Gestion des ressources forestières et des milieux naturels
Foucher, Samuel
Lejeune, Philippe ; Université de Liège - ULiège > Département GxABT > Gestion des ressources forestières et des milieux naturels
Linchant, Julie ; Université de Liège - ULiège > TERRA Research Centre
Théau, Jérôme
Language :
English
Title :
Multispecies detection and identification of African mammals in aerial imagery using convolutional neural networks
This dataset contains aerial images, model result files and the code used in the paper: "Multispecies detection and identification of African mammals in aerial imagery using convolutional neural networks". It is divided into 3 main parts: 1) The code, which contains the mmdetection v1.0.0 package as well as the adapted version and Jupyter notebooks; 2) The data ('general_dataset.zip'), which contains all the images and annotations of the general dataset used for training the deep learning models in the paper; 3) The results ('results_files.zip'), which contain the models' outputs on the general dataset.
Almond, G.M. & Petersen, T. (2020) Living planet report 2020 - bending the curve of biodiversity loss. Gland, Switzerland: WWF.
Barbedo, J.G.A., Koenigkan, L.V., Santos, T.T. & Santos, P.M. (2019) A study on the detection of cattle in UAV images using deep learning. Sensors, 19(24), 5436. https://doi.org/10.3390/s19245436
Beery, S., Liu, Y., Morris, D., Piavis, J., Kapoor, A., Joshi, N. et al. (2020) Synthetic examples improve generalization for rare classes. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision. Snowmass, CO: Institute of Electrical and Electronics Engineers (IEEE), pp. 852–862. https://doi.org/10.1109/WACV45572.2020.9093570
Ceballos, G., Ehrlich, P.R., Barnosky, A.D., Garcïa, A., Pringle, R.M. & Palmer, T.M. (2015) Accelerated modern human-induced species losses: entering the sixth mass extinction. Science Advances, 1(5), e1400253. https://doi.org/10.1126/sciadv.1400253
Chabot, D. & Bird, D.M. (2015) Wildlife research and management methods in the 21st century: where do unmanned aircraftt in? Journal of Unmanned Vehicle Systems, 3(4), 137–155. https://doi.org/10.1139/juvs-2015-0021
Chen, K., Wang, J., Pang, J., Cao, Y., Xiong, Y., Li, X. et al. (2019) MMDetection: open MMLab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155.
Craigie, I., Baillie, J., Balmford, A., Carbone, C., Collen, B., Green, R. et al. (2010) Large mammal population declines in Africa's protected areas. Biological Conservation, 143, 2221–2228. https://doi.org/10.1016/j.biocon.2010.06.007
Eikelboom, J.A., Wind, J., van de Ven, E., Kenana, L.M., Schroder, B., de Knegt, H.J. et al. (2019) Improving the precision and accuracy of animal population estimates with aerial image object detection. Methods in Ecology and Evolution, 10(11), 1875–1887. https://doi.org/10.1111/2041-210X.13277.
Everingham, M., Van Gool, L., Williams, C.K., Winn, J. & Zisserman, A. (2010) The pascal visual object classes (VOC) challenge. International Journal of Computer Vision, 88(2), 303–338. https://doi.org/10.1007/s11263-014-0733-5
Gaidet-Drapier, N., Fritz, H., Bourgarel, M., Renaud, P.-C., Poilecot, P., Chardonnet, P. et al. (2006) Cost and efficiency of large mammal census techniques: comparison of methods for a participatory approach in a communal area, Zimbabwe. Biodiversity and Conservation, 15(2), 735–754. https://doi.org/10.1007/s10531-004-1063-7
He, K., Zhang, X., Ren, S. & Sun, J. (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 770–778. https://doi.org/10.1109/CVPR.2016.90
Hetem, R.S., Fuller, A., Maloney, S.K. & Mitchell, D. (2014) Responses of large mammals to climate change. Temperature, 1(2), 115–127. https://doi.org/10.4161/temp.29651
IPCC. (2014) Climate change 2014: synthesis report. In: Pachauri, R. & Meyer, L. (Eds.) Contribution of working groups I, II and III to the fifth assessment report of the intergovernmental panel on climate change. Geneva, Switzerland: IPCC, p. 151.
Isbell, F., Gonzalez, A., Loreau, M., Cowles, J., Dıaz, S., Hector, A. et al. (2017) Linking the influence and dependence of people on biodiversity across scales. Nature, 546(7656), 65–72. https://doi.org/10.1038/nature22899
Jachmann, H. (1991) Evaluation of four survey methods for estimating elephant densities. African Journal of Ecology, 29, 188–195. https://doi.org/10.1111/j.1365-2028.1991.tb01001.x
Kellenberger, B., Marcos, D. & Tuia, D. (2018) Detecting mammals in UAV images: best practices to address a substantially imbalanced dataset with deep learning. Remote Sensing of Environment, 216, 139–153. https://doi.org/10.1016/j.rse.2018.06.028
Lacher, T.E., Davidson, A.D., Fleming, T.H., Gómez-Ruiz, E.P., McCracken, G.F., Owen-Smith, N. et al. (2019) The functional roles of mammals in ecosystems. Journal of Mammalogy, 100(3), 942–964. https://doi.org/10.1093/jmammal/gyy183
Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B. & Belongie, S. (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 936–944. https://doi.org/10.1109/CVPR.2017.106
Lin, T.-Y., Goyal, P., Girshick, R., He, K. & Dollár, P. (2017) Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 2999–3007. https://doi.org/10.1109/ICCV.2017.324
Linchant, J., Lhoest, S., Quevauvillers, S., Lejeune, P., Vermeulen, C., Ngabinzeke, J.S. et al. (2018) UAS imagery reveals new survey opportunities for counting hippos. PLoS One, 13(11), 1–17. https://doi.org/10.1371/journal.pone.0206413
Linchant, J., Lhoest, S., Quevauvillers, S., Semeki, J., Lejeune, P. & Vermeulen, C. (2015) WIMUAS: developing a tool to review wildlife data from various UAS flight plans. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 40, 379–384. https://doi.org/10.5194/isprsarchives-XL-3-W3-379-2015
Linchant, J., Lisein, J., Ngabinzeke, J., Lejeune, P. & Vermeulen, C. (2015) Are unmanned aircraft systems (UAS) the future of wildlife monitoring? A review of accomplishments and challenges. Mammal Review, 45, 239–252. https://doi.org/10.1111/mam.12046
Mayaux, P., Bartholomé, E., Fritz, S. & Belward, A. (2004) A new land-cover map of Africa for the year 2000. Journal of Biogeography, 31(6), 861–877. https://doi.org/10.1111/j.1365-2699.2004.01073.x
Moreni, M., Theau, J. & Foucher, S. (2021) Train fast while reducing false positives: improving animal classification performance using convolutional neural networks. Geomatics, 1(1), 34–49. https://doi.org/10.3390/geomatics1010004
Mulero-Pázmány, M., Stolper, R., Van Essen, L., Negro, J.J. & Sassen, T. (2014) Remotely piloted aircraft systems as a rhinoceros antipoaching tool in Africa. PLoS One, 9(1), 1–10. https://doi.org/10.1371/journal.pone.0083873
Naudé, J. & Joubert, D. (2019a) The aerial elephant dataset: a new public benchmark for aerial object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 48–55.
Naudé, J. & Joubert, D. (2019b) Data from: the aerial elephant dataset: a new public benchmark for aerial object detection. Zenodo. https://doi.org/10.5281/zenodo.3234780
Oksuz, K., Cam, B.C., Kalkan, S. & Akbas, E. (2020) Imbalance problems in object detection: a review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1. https://doi.org/10.1109/TPAMI.2020.2981890
Pang, J., Chen, K., Shi, J., Feng, H., Ouyang, W. & Lin, D. (2019) Libra R-CNN: towards balanced learning for object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 821–830. https://doi.org/10.1109/CVPR.2019.00091
Peng, J., Wang, D., Liao, X., Shao, Q., Sun, Z., Yue, H. et al. (2020) Wild animal survey using UAS imagery and deep learning: modified Faster R-CNN for kiang detection in Tibetan Plateau. ISPRS Journal of Photogrammetry and Remote Sensing, 169, 364–376. https://doi.org/10.1016/j.isprsjprs.2020.08.026
Ren, S., He, K., Girshick, R. & Sun, J. (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031
Rey, N., Volpi, M., Joost, S. & Tuia, D. (2017) Detecting animals in African Savanna with UAVs and the crowds. Remote Sensing of Environment, 200, 341–351. https://doi.org/10.1016/j.rse.2017.08.026
Ribera, J., Guera, D., Chen, Y. & Delp, E.J. (2019) Locating objects without bounding boxes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6479–6489.
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S. et al. (2015) Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3), 211–252. https://doi.org/10.1007/s11263-015-0816-y
Soviany, P. & Ionescu, R.T. (2018) Optimizing the trade-off between single-stage and two-stage deep object detectors using image difficulty prediction. In Proceedings of the 20th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing. pp. 209–214. https://doi.org/10.1109/SYNASC.2018.00041
Thuiller, W., Broennimann, O., Hughes, G., Alkemade, J.R.M., Midgley, G.F. & Corsi, F. (2006) Vulnerability of African mammals to anthropogenic climate change under conservative land transformation assumptions. Global Change Biology, 12(3), 424–440. https://doi.org/10.1111/j.1365-2486.2006.01115.x
Vermeulen, C., Lejeune, P., Lisein, J., Sawadogo, P. & Bouché, P. (2013) Unmanned aerial survey of elephants. PLoS One, 8(2), 1–7. https://doi.org/10.1371/journal.pone.0054700
Watts, A.C., Perry, J.H., Smith, S.E., Burgess, M.A., Wilkinson, B.E., Szantoi, Z. et al. (2010) Small unmanned aircraft systems for low-altitude aerial surveys. The Journal of Wildlife Management, 74(7), 1614–1619. https://doi.org/10.1111/j.1937-2817.2010.tb01292.x
Witmer, G.W. (2005) Wildlife population monitoring: some practical considerations. Wildlife Research, 32(3), 259–263. https://doi.org/10.1071/WR04003
Xu, B., Wang, W., Falzon, G., Kwan, P., Guo, L., Chen, G. et al. (2020) Automated cattle counting using Mask R-CNN in quadcopter vision system. Computers and Electronics in Agriculture, 171, 105300. https://doi.org/10.1016/j.compag.2020.105300
Zhao, Z.-Q., Zheng, P., Xu, S.-T. & Wu, X. (2019) Object detection with deep learning: a review. IEEE Transactions on Neural Networks and Learning Systems, 30(11), 3212–3232. https://doi.org/10.1109/TNNLS.2018.2876865
Zhu, X.X., Tuia, D., Mou, L., Xia, G.-S., Zhang, L., Xu, F. et al. (2017) Deep learning in remote sensing: a comprehensive review and list of resources. IEEE Geoscience and Remote Sensing Magazine, 5(4), 8–36. https://doi.org/10.1109/MGRS.2017.2762307
