Agronomy and Crop Science; Plant Science; Horticulture
Abstract :
[en] Appropriate statistical analysis of the validation data for diagnostic tests facilitates the evaluation of the performance criteria and increases the confidence in the conclusions drawn from these data. A comprehensive approach to analysing and reporting data from validation studies and inter-laboratory comparisons such as test performance studies is described. The proposed methods, including statistical analyses, presentation and interpretation of the data, are illustrated using a real dataset generated during a test performance study conducted in the framework of the European project, VALITEST. This analytical approach uses, wherever possible and whenever applicable, statistical analyses recommended by international standards illustrating their application to plant health diagnostic tests. The present work is addressed to plant health diagnosticians and researchers interested and/or involved in the validation of plant diagnostic tests, and also aims to convey the necessary information to those without a statistical background. Detailed statistical explanations are provided in the Appendices.
Disciplines :
Biochemistry, biophysics & molecular biology
Author, co-author :
Massart, Sébastien ; Université de Liège - ULiège > TERRA Research Centre > Gestion durable des bio-agresseurs
Lebas, Benedicte; Gembloux Agro-Bio Tech, TERRA, Laboratory of Plant Pathology, University of Liège, Liège, Belgium
Chabirand, Aude; Pests and Tropical Pathogens Unit, ANSES Plant Health Laboratory, Saint Pierre, France
Chappé, Anne-Marie; Nematology Unit, ANSES Plant Health Laboratory, Le Rheu, France
Dreo, Tanja; National Institute of Biology, Ljubljana, Slovenia
Faggioli, Francesco; Consiglio per la Ricerca in Agricoltura e l'analisi dell'economia agraria – Centro di Ricerca Difesa e Certificazione, Roma, Italy
Harrison, Catherine; Fera Science Ltd, York, United Kingdom
Macarthur, Roy; Fera Science Ltd, York, United Kingdom
Mehle, Natasha; National Institute of Biology, Ljubljana, Slovenia ; School for Viticulture and Enology, University of Nova Gorica, Vipava, Slovenia
Mezzalama, Monica; Department of Agricultural Forest and Food Sciences and AGROINNOVA – Centre of Competence for the Innovation in the Agro-environmental Sector, University of Torino, Turin, Italy
Petter, Françoise; European and Mediterranean Plant Protection Organization, Paris, France
Ravnikar, Maja; National Institute of Biology, Ljubljana, Slovenia
Renvoisé, Jean-Philippe; Quarantine Unit, ANSES, Plant Health Laboratory, Clermont-Ferrand, France
Spadaro, Davide; Department of Agricultural Forest and Food Sciences and AGROINNOVA – Centre of Competence for the Innovation in the Agro-environmental Sector, University of Torino, Turin, Italy
Tomassoli, Laura; Consiglio per la Ricerca in Agricoltura e l'analisi dell'economia agraria – Centro di Ricerca Difesa e Certificazione, Roma, Italy
Tomlinson, Jenny; Fera Science Ltd, York, United Kingdom
Trontin, Charlotte; Quarantine Unit, ANSES, Plant Health Laboratory, Clermont-Ferrand, France
van der Vlugt, René; Wageningen University and Research, Wageningen, Netherlands
Vučurović, Ana; National Institute of Biology, Ljubljana, Slovenia
Weekes, Rebecca; Department of Agricultural Forest and Food Sciences and AGROINNOVA – Centre of Competence for the Innovation in the Agro-environmental Sector, University of Torino, Turin, Italy
Brostaux, Yves ; Université de Liège - ULiège > TERRA Research Centre > Modélisation et développement
From the Division of Allergy and Immunology, Department of Medicine (CGB, GMS), and the Department of Pathology (SF), Come, University Medical College, and the Division of Rheumatologya nd Clinical Immunology, Departmento f Medicine( NC), North Shore University Hospital, Cornell University Medical College, New York, New York. This work was supported in part by grants from the NationalI nstitutes of Health, United States Public Health Service( AI \].9080a nd A110811), and the Hyde and Watson Foundation. Requests for reprints should be addressed to Gillian M. Shepherd, M.D., The New York HospitaI-CornellM edicalC enter, 525 East6 8th Street, New York, New York 1002\]. Manuscript submitted January 25, 1989, and accepted in revised form May 18, 1989.
Agresti A, Coull B A (1998) Approximate is better than “Exact” for interval estimation of binomial proportions. American Statistician 52, 119–126.
Chabirand A, Loiseau M, Renaudin I, Poliakoff F (2017) Data processing of qualitative results from an interlaboratory comparison for the detection of ‘Flavescence dorée’ phytoplasma: how the use of statistics can improve the reliability of the method validation process in plant pathology. PLoS ONE 12(4), e0175247, https://doi.org/10.1371/journal.pone.0175247
EPPO (2014) PM7/122 (1) Guidelines for the organization of interlaboratory comparisons by plant pest diagnostic laboratories. Bulletin OEPP/EPPO Bulletin 44(3), 390–399, https://onlinelibrary.wiley.com/doi/epdf/10.1111/epp.12162.
EPPO (2019) PM 7/98 (4) Specific requirements for laboratories preparing accreditation for a plant pest diagnostic activity. Bulletin OEPP/EPPO Bulletin 49(3), 530–563, https://onlinelibrary.wiley.com/doi/epdf/10.1111/epp.12629.
EPPO (2021) PM 7/147 (1) Guidelines for the production of biological reference material. Bulletin OEPP/EPPO Bulletin 51(3), 499–506, https://onlinelibrary.wiley.com/doi/full/10.1111/epp.12781.
Fisher R A (1922) On the interpretation of χ2 from contingency tables, and the calculation of P. Journal of the Royal Statistical Society 85(1), 87–94, https://doi.org/10.2307/2340521.
Fleiss JL, Levin B, Paik MC (2003) Statistical analyses for rates and proportions. Third Edition. John Wiley & Sons. New York.
Franco Ortega S, del Pilar Bustos López M, Nari L, Boonham N, Gullino ML, Spadaro D (2021) Rapid detection of Monilinia fructicola and Monilinia laxa on peaches and nectarines using loop-mediated isothermal amplification. Plant Disease 103, 2305–2314, https://apsjournals.apsnet.org/doi/pdfplus/10.1094/PDIS-01-19-0035-RE
Franco Ortega S, Prencipe S, Gullino ML, Spadaro D (2020) New molecular tool for a quick and easy detection of apple scab in the field. Agronomy 10 (4), 581, https://doi.org/10.3390/agronomy10040581.
Glas AS, Lijmer JG, Prins MH, Bonsel GJ, Bossuyt PM (2003) The diagnostic odds ratio: a single indicator of test performance. Journal of Clinical Epidemiology 56(11), 1129–1135.
Hess AS, Shardell M, Johnson JK, Thom KA, Strassle P, Netzer G, Harris AD (2012) Methods and recommendations for evaluating and reporting a new diagnostic test. European Journal of Clinical Microbiology and Infectious Diseases 31(9), 2111–2116.
ISF (2020) ISHI-Veg guidelines for the validation of seed health methods, version 3, https://www.worldseed.org/wp-content/uploads/2020/12/MVGuidelines_v3_November-2020.pdf.
ISO 16140-2 (2016) Microbiology of food chain – method validation – Part 2: Protocol for the validation of alternative (proprietary) methods against a reference method. International organization for standardization, Geneva, Switzerland.
ISTA (2019) Procedure ‘validation methods and organizing and analyzing results of interlaboratory comparative tests (CT)’, https://www.seedtest.org/upload/cms/user/TCOM-P-10-ValidatingmethodsandresultsofCTsV1.141.pdf.
Langton SD, Chevennement R, Nagelkerke N, Lombard B (2002) Analysing collaborative trials for qualitative microbiological methods: accordance and concordance. International Journal of Food Microbiology 79, 175–181.
McCullagh P, Nelder JA (1989) Generalized Linear Models, 2nd revised edition, London: Chapman & Hall.
Massart S, Brostaux Y, Brabarossa L, Batlle A, César V, Dutrecq O, Fonseca F, Guillem R, Komorowska B, Olmos A, Steyer S, Wetzel T, Kummert J, Jijakli M H (2009a) Inter-laboratory evaluation of two reverse-transcriptase polymeric chain reaction-based methods for the detection of four fruit tree viruses. Annals of Applied Biology 154, 133–141.
Massart S, Brostaux Y, Brabarossa L, César V, Cieslinska M, Dutrecq O, Fonseca F, Guillem R, Laviña A, Olmos A, Steyer S, Wetzel T, Kummert J, Jijakli MH (2009b) Inter-laboratory evaluation of a duplex RT-PCR method using crude extracts for the simultaneous detection of Prune dwarf virus and Prunus necrotic ringspot virus. European Journal of Plant Pathology 122, 539–547.
NATA (2018) Guidelines for the Validation and Verification of Quantitative and Qualitative Test Methods, in: Technical Note 17. Canberra, Australia, https://www.nata.com.au/phocadownload/gen-accreditation-guidance/Validation-and-Verification-of-Quantitative-and-Qualitative-Test-Methods.pdf.
Newcombe RG (1998) Two-sided confidence intervals for the single proportion: comparison of seven methods. Statistics in Medicine 17, 857–872. https://doi.org/10.1002/(SICI)1097-0258(19980430)17:8<857::AID-SIM777>3.0.CO;2-E.
Parikh R, Parikh S, Arun E, Thomas R (2009) Likelihood ratios: clinical application in day-to-day practice. Indian Journal of Ophtalmology 57(3), 217–221.
Renvoisé JP, Chambon F, Gleize M, Pradeilles N, Garnier S, Rolland M (2019) Selection, optimization and characterization of molecular tests for the detection of Tobacco ringspot virus (TRSV). Bulletin OEPP/EPPO Bulletin 49(1), 111–121.
Schiettecatte J, Anckaert E, Smitz J (2012) Interferences in immunoassays. In: Advances in Immunoassay Technology. Chiu NHL, Christopoulos TK (eds). IntechOpen, ISBN 978-953-51-0440-7, 10.5772/1967. https://www.intechopen.com/chapters/33740
Simel DL, Samsa GP, Matchar DB (1991) Likelihood ratios with confidence: sample size estimation for diagnostic test studies. Journal of Clinical Epidemiology 44(8), 763–770.
Šimundić, AM (2009). Measures of diagnostic accuracy: basic definitions. The Journal of the International Federation of Clinical Chemistry and Laboratory Medicine 19(4), 203–211.
Trontin C, Agstner B, Altenbach D, Anthoine G, Bagińska H, Brittain I, Chabirand A, Chappé AM, Dahlin P, Dreo T, Freye-Minks C, Gianinazzi C, Harrison C, Jones G, Luigi M, Massart S, Mehle N, Mezzalama M, Mouaziz H, Petter F, Ravnikar M, Raaymakers T M, Renvoisé J P, Rolland M, Santos Paiva M, Seddas S, van der Vlugt R, Vučurović A (2021) VALITEST: Validation of diagnostic tests to support plant health. Bulletin OEPP/EPPO Bulletin 51(1), 198–206.
Wehling P, LaBudde RA, Brunelle SL, Nelson MT (2011) Probability of detection (POD) as a statistical model for the validation of qualitative methods. Journal of AOAC International 94(1), 335–347.
