Cloud architecture for plant phenotyping research

Debauche, Olivier; Mahmoudi, Sidi Ahmed; De Cock, Nicolas; Mahmoudi, Saïd; Manneback, Pierre; Lebeau, Frédéric

doi:10.1002/cpe.5661

Download

Article (Scientific journals)

Cloud architecture for plant phenotyping research

Debauche, Olivier; Mahmoudi, Sidi Ahmed; De Cock, Nicolas et al.

2020 • In Concurrency and Computation: Practice and Experience

Peer Reviewed verified by ORBi

Permalink
https://hdl.handle.net/2268/243101

DOI
10.1002/cpe.5661

Files (1)Send to Details Statistics Bibliography Similar publications

Files

Full Text

16649681.pdf

Author preprint (7.27 MB)

Download

All documents in ORBi are protected by a user license.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Keywords :

cloud architecture; digital phenotyping; lambda architecture; plant phenotyping; research application hosting platform

Abstract :

[en] Digital phenotyping is an emergent sciencemainly based on imagery techniques. The tremendous amount of data generated needs important cloud computing for their processing. The coupling of recent advance of distributed databases and cloud computing offers new possibilities of big data management and data sharing for the scientific research. In this paper, we present a solution combining a lambda architecture built around Apache Druid and a hosting platform leaning on Apache Mesos. Lambda architecture has already proved its performance and robustness. However, the capacity of ingesting and requesting of the database is essential and can constitute a bottleneck for the architecture, in particular, for in terms of availability and response time of data. We focused our experimentation on the response time of different databases to choose the most adapted for our phenotyping architecture. Apache Druid has shown its ability to respond to typical queries of phenotyping applications in times generally inferior to the second.

Disciplines :

Computer science
Phytobiology (plant sciences, forestry, mycology...)

Author, co-author :

Debauche, Olivier ; Université de Liège - ULiège > Terra

Mahmoudi, Sidi Ahmed

De Cock, Nicolas ; Université de Liège - ULiège > Département GxABT > Biosystems Dynamics and Exchanges

Mahmoudi, Saïd

Manneback, Pierre

Lebeau, Frédéric ; Université de Liège - ULiège > Département GxABT > Biosystems Dynamics and Exchanges

Language :

English

Title :

Cloud architecture for plant phenotyping research

Publication date :

08 January 2020

Journal title :

Concurrency and Computation: Practice and Experience

ISSN :

1532-0626

eISSN :

1532-0634

Publisher :

John Wiley & Sons, Hoboken, United States - New Jersey

Peer reviewed :

Peer Reviewed verified by ORBi

Available on ORBi :

since 14 January 2020

Statistics

Number of views

98 (16 by ULiège)

Number of downloads

317 (11 by ULiège)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

Bibliography

Singh A, Ganapathysubramanian B, Singh AK, Sarkar S. Machine learning for high-throughput stress phenotyping in plants. Trends Plant Sci. 2016;21(2):110-124.
Tisné S, Serrand Y, Bach L, et al. Phenoscope: an automated large-scale phenotyping platform offering high spatial homogeneity. Plant J. 2013;74(3):534-544.
Virlet N, Sabermanesh K, Sadeghi-Tehran P, Hawkesford MJ. Field scanalyzer: an automated robotic field phenotyping platform for detailed crop monitoring. Funct Plant Biol. 2017;44(1):143-153.
Andrade-Sanchez P, Gore MA, Heun JT, et al. Development and evaluation of a field-based high-throughput phenotyping platform. Funct Plant Biol. 2014;41(1):68-79.
Marshall-Colon A, Long SP, Allen DK, et al. Crops in silico: generating virtual crops using an integrative and multi-scale modeling platform. Front Plant Sci. 2017;8:786.
Wang L, Ware D, Lushbough C, Merchant N, Stein L. A genome-wide association study platform built on iPlant cyber-infrastructure. Concurrency Computat Pract Exper. 2015;27(2):420-432.
Xu H, Russell T, Coposky J, et al. iRODS primer 2: integrated rule-oriented data system. Synth Lect Inf Concepts Retr Serv. 2017;9(3):1-131.
Ananthakrishnan R, Chard K, Foster I, Tuecke S. Globus platform-as-a-service for collaborative science applications. Concurrency Computat Pract Exper. 2015;27(2):290-305.
Allcock W, Bresnahan J, Kettimuthu R, et al. The Globus striped GridFTP framework and server. In: Proceedings of the 2005 ACM/IEEE Conference on Supercomputing; 2005; Seattle, WA.
Liu B, Sotomayor B, Madduri R, Chard K, Foster I. Deploying bioinformatics workflows on clouds with galaxy and globus provision. In: Proceedings of the 2012 SC Companion: High Performance Computing, Networking Storage and Analysis; 2012; Salt Lake City, UT.
Bhuvaneshwar K, Sulakhe D, Gauba R, et al. A case study for cloud based high throughput analysis of NGS data using the globus genomics system. Comput Struct Biotechnol J. 2015;13:64-74.
Caballer M, Segrelles D, Moltó G, Blanquer I. A platform to deploy customized scientific virtual infrastructures on the cloud. Concurrency Computat Pract Exper. 2015;27(16):4318-4329.
Zaharia M, Xin RS, Wendell P, et al. Apache Spark: a unified engine for big data processing. Commun ACM. 2016;59(11):56-65.
Yan Y, Huang L. Large-scale image processing research cloud. Cloud Computing. 2014:88-93.
Brazma A, Hingamp P, Quackenbush J, et al. Minimum information about a microarray experiment (MIAME)-toward standards for microarray data. Nature Genetics. 2001;29:365-371.
Taylor CF, Panton NW, Lilley KS, et al. The minimum information about a proteomics experiment (MIAPE). Nat Biotechnol. 2007;25(8):887-893.
Fiehn O, Robertson D, Griffin J, et al. The metabolomics standards initiative (MSI). Metabolomics. 2007;3(3):175-178.
Yilmaz P, Kottmann R, Field D, et al. Minimum information about a marker gene sequence (MIMARKS) and minimum information about any (x) sequence (MIxS) specifications. Nature Biotechnology. 2011;29:415-420.
Zimmermann P, Schildknecht B, Craigon D, et al. MIAME/plant – adding value to plant microarrray experiments. Plant Methods. 2006;2(1). Article No. 1.
Fiehn O, Sumner LW, Rhee SY, et al. Minimum reporting standards for plant biology context information in metabolomic studies. Metabolomics. 2007;3(3):195-201.
Morrisson N, Bearden D, Bundy J, et al. Standard reporting requirements for biological samples in metabolomics experiments: environmental context. Metabolomics. 2007;3(3):203-210.
Krajewski P, Chen D, Ćwiek H, et al. Towards recommendations for metadata and data handling in plant phenotyping. J Exp Bot. 2015;66(18):5417-5427.
Debauche O, Mahmoudi S, Andriamandroso ALH, Manneback P, Bindelle J, Lebeau F. Web-based cattle behavior service for researchers based on the smartphone inertial central. Procedia Comput Sci. 2017;110:110-116.
Debauche O, Mahmoudi S, Andriamandroso ALH, Manneback P, Bindelle J, Lebeau F. Cloud services integration for farm animals' behavior studies based on smartphones as activity sensors. J Ambient Intell Humaniz Comput. 2018.
Debauche O, El Moulat M, Mahmoudi S, Boukraa S, Manneback P, Lebeau F. Web monitoring of bee health for researchers and beekeepers based on the internet of things. Procedia Comput Sci. 2018;130:991-998.
Debauche O, El Moulat M, Mahmoudi S, Manneback P, Lebeau F. Irrigation pivot-center connected at low cost for the reduction of crop water requirements. In: Proceedings of the International Conference on Advanced Communication Technologies and Networking (CommNet 2018); 2018; Marrakech, Morocco.
El Moulat M, Debauche O, Mahmoudi S, Aït Brahim L, Manneback P, Lebeau F. Monitoring system using Internet of Things for potential landslides. Procedia Comput Sci. 2018;134:26-34.
Debauche O, Mahmoudi S, Manneback P, et al. Cloud architecture for digital phenotyping and automation. In: Proceedings of the 3rd International Conference on Cloud Computing Technologies and Applications (CloudTech'17); 2018; Rabat, Morocco.
Marz N, Warren J. Big Data: Principles and Best Practices of Scalable Real-Time Data Systems. New York, NY: Manning Publications; 2015.
Shvachko K, Kuang H, Radia S, Chansler R. The Hadoop distributed file system. In: Proceedings of the 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST); 2010; Incline Village, NV.
Yang F, Tschetter E, Léauté X, Ray N, Merlino G, Ganguli D. Druid: a real-time analytical data store. In: Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data; 2014; Snowbird, UT.
Yang F, Merlino G, Ray N, Léauté X, Gupta H, Tschetter E. The RADStack: open source lambda architecture for interactive analytics. In: Proceedings of the 50th Hawaii International Conference on System Sciences; 2017; Waikoloa Village, HI.
Tschetter E. Introducing druid: real-time analytics at a billion rows per second. 2011. Accessed June 13, 2017. http://druid.io/blog/2011/04/30/introducing-druid.html
Díaz M, Martín B, Rubio B. State-of-the-art, challenges, and open issues in the integration of internet of things and cloud computing. J Netw Comput Appl. 2016;67:99-117.
Bai G, Ge Y, Hussain P, Baenziger PS, Grae G. A multi-sensor system for high throughput field phenotyping in soybean and wheat breeding. Comput Electron Agric. 2016;128:181-192.
Hindman B, Konwinski A, Zaharia M, et al. Mesos: a platform for fine-grained resource sharing in the data center. In: Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation; 2011; Boston, MA.
Merkel D. Docker: lightweight linux containers for consistent development and deployment. Linux Journal. 2014;2014(239).
Klukas C, Chen D, Pape J-M. Integrated analysis platform: an open-source information system for high-throughput plant phenotyping. Plant Physiology. 2014;165(2):506-518.
Cuzzocrea A, Moussa R, Vercelli G. An innovative lambda-architecture-based data warehouse maintenance framework for effective and efficient near-real-time OLAP over big data. In: Chin FYL, Chen CLP, Khan L, Lee K, Zhang L-J, eds. Big Data – BigData 2018: 7th International Congress, Held as Part of the Services Conference Federation, SCF 2018, Seattle, WA, USA, June 25-30, 2018, Proceedings. Cham, Switzerland: Springer; 2018:149-165.
Lee D, Choi J, Kim J-H, et al. LRFU: a spectrum of policies that subsumes the least recently used and least frequently used policies. IEEE Trans Comput. 2001;50(12):1352-1361.
Zawodny J. Redis: lightweight key/value store that goes the extra mile. Linux Magazine. 2009;79.
Abadi M, Barham P, Chen J, et al. TensorFlow: a system for large-scale machine learning. In: Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI); 2016; Savannah, GA.
Gropp W, Lusk E, Doss N, Skjellum A. A high-performance, portable implementation of the MPI message passing interface standard. Parallel Computing. 1996;22(6):789-828.
Foundation Apache. Aurora is a Mesos framework for long-running services and cron jobs. 2017. Accessed June 7, 2017. http://aurora.apache.org
Lecron F, Mahmoudi SA, Benjelloun M, Mahmoudi S, Manneback P. Heterogeneous computing for vertebra detection and segmentation in X-ray images. Int J Biomed Imaging. 2011;2011.
Mahmoudi SA, Belarbi MA, Mahmoudi S, Belalem G. Towards a smart selection of resources in the cloud for low-energy multimedia processing. Concurrency Computation. 2018;30(12).
Mahmoudi SA, Manneback P. Multi-CPU/multi-GPU based framework for multimedia processing. In: Computer Science and Its Applications: 5th IFIP TC 5 International Conference, CIIA 2015, Saida, Algeria, May 20-21, 2015, Proceedings. Cham, Switzerland: Springer; 2015:54-65. IFIP Advances in Information and Communication Technology; vol. 456.
Mahmoudi SA, Manneback P. Multi-GPU based event detection and localization using high definition videos. In: Proceedings of the 2014 International Conference on Multimedia Computing and Systems (ICMCS); 2014; Marrakech, Morocco.
Mahmoudi SA, Ammar M, Joris GL, Abbou A. Real time GPU-based segmentation and tracking of the left ventricle on 2D echocardiography. In: Bioinformatics and Biomedical Engineering: 4th International Conference, IWBBIO 2016, Granada, Spain, April 20-22, 2016, Proceedings. Cham, Switzerland: Springer; 2016:602-614. Lecture Notes in Computer Science; vol. 9656.
da Cunha Possa P, Mahmoudi SA, Harb N, Valderrama C. A new self-adapting architecture for feature detection. In: Proceedings of the 22nd International Conference on Field Programmable Logic and Applications (FPL); 2012; Oslo, Norway.
Council Transaction Processing. TPC-H benchmark. 2013. Accessed February 1, 2019. http://tpc.org/tpc
Léauté X. Benchmarking Druid. 2014. Accessed January 8, 2019. http://druid.io/blog/2014/03/17/benchmarking-druid.html