Computer Simulation; Drug Development/legislation & jurisprudence/methods; Humans; Models, Theoretical; Risk Assessment/methods; Terminology as Topic
Abstract :
[en] The value of in silico methods in drug development and evaluation has been demonstrated repeatedly and convincingly. While their benefits are now unanimously recognized, international standards for their evaluation, accepted by all stakeholders involved, are still to be established. In this white paper, we propose a risk-informed evaluation framework for mechanistic model credibility evaluation. To properly frame the proposed verification and validation activities, concepts such as context of use, regulatory impact and risk-based analysis are discussed. To ensure common understanding between all stakeholders, an overview is provided of relevant in silico terminology used throughout this paper. To illustrate the feasibility of the proposed approach, we have applied it to three real case examples in the context of drug development, using a credibility matrix currently being tested as a quick-start tool by regulators. Altogether, this white paper provides a practical approach to model evaluation, applicable in both scientific and regulatory evaluation contexts.
Disciplines :
Engineering, computing & technology: Multidisciplinary, general & others
Author, co-author :
Musuamba, Flora T ; EMA Modelling and Simulation Working Party, Amsterdam, The Netherlands. ; Federal Agency for Medicines and Health Products, Brussels, Belgium. ; Faculté des Sciences Pharmaceutiques, Université de Lubumbashi, Lubumbashi
Skottheim Rusten, Ine ; EMA Modelling and Simulation Working Party, Amsterdam, The Netherlands. ; Norvegian Medicines Agency, Oslo, Norway.
Lesage, Raphaëlle; Biomechanics Section, KU Leuven, Leuven, Belgium. ; Virtual Physiological Human Institute, Leuven, Belgium.
Russo, Giulia; Department of Drug and Health Sciences, University of Catania, Catania, Italy.
Wangorsch, Gaby; EMA Modelling and Simulation Working Party, Amsterdam, The Netherlands. ; Paul-Ehrlich-Institut (Federal Institute for Vaccines and Biomedicines), Langen,
Manolis, Efthymios; EMA Modelling and Simulation Working Party, Amsterdam, The Netherlands. ; European Medicines Agency, Amsterdam, The Netherlands.
Karlsson, Kristin E ; EMA Modelling and Simulation Working Party, Amsterdam, The Netherlands. ; Swedish Medical Products Agency, Uppsala, Sweden.
Kulesza, Alexander; Novadiscovery, Lyon, France.
Courcelles, Eulalie; Novadiscovery, Lyon, France.
Boissel, Jean-Pierre; Novadiscovery, Lyon, France.
Rousseau, Cécile F; Voisin Consulting Life Sciences, Boulogne (Paris), France.
Voisin, Emmanuelle M ; Voisin Consulting Life Sciences, Boulogne (Paris), France.
Alessandrello, Rossana; AQuAS - Agency for Health Quality and Assessment of Catalonia, Catalonia, Spain.
Curado, Nuno; Exploristics, Belfast, UK.
Dall'ara, Enrico; Insigneo Institute, Sheffield University, Sheffield, UK.
Rodriguez, Blanca ; Department of Computer Science, British Heart Foundation Centre of Research
Pappalardo, Francesco ; Department of Drug and Health Sciences, University of Catania, Catania, Italy.
Geris, Liesbet ; Université de Liège - ULiège > GIGA > GIGA In silico medecine - Biomechanics Research Unit ; Biomechanics Section, KU Leuven, Leuven, Belgium. ; Virtual Physiological Human Institute, Leuven, Belgium.
Manolis E, Brogren J, Cole S, et al. Commentary on the MID3 Good Practices Paper. CPT Pharmacometrics Syst. Pharmacol. 2017;6:416-417. https://doi.org/10.1002/psp4.12223.
Pappalardo F, Russo G, Musuamba FT, Viceconti M. In silico clinical trials: concepts and early adoptions. Brief Bioinform. 2019;20(5):1699-1708. https://doi.org/10.1093/bib/bby043.
Wang Y, Zhu H, Madabushi R, Liu Q, Huang SM, Zineh I. Model-informed drug development: current us regulatory practice and future considerations. Clin Pharmacol Ther. 2019;105(4):899-911. https://doi.org/10.1002/cpt.1363.
EFPIA MID3 Workgroup, Marshall SF, Burghaus R, et al. Good practices in model-informed drug discovery and development: practice, application, and documentation. CPT Pharmacometrics Syst Pharmacol. 2016;5(3):93-122. https://doi.org/10.1002/psp4.12049.
Luzon E, Blake K, Cole S, Nordmark A, Versantvoort C, Berglund EG. Physiologically based pharmacokinetic modeling in regulatory decision-making at the European Medicines Agency. Clin Pharmacol Ther. 2017;102(1):98-105. https://doi.org/10.1002/cpt.539.
Zhao P, Rowland M, Huang SM. Best practice in the use of physiologically based pharmacokinetic modeling and simulation to address clinical pharmacology regulatory questions. Clin Pharmacol Ther. 2012;92(1):17-20. https://doi.org/10.1038/clpt.2012.68.
Madabushi R, Wang Y, Zineh I. A holistic and integrative approach for advancing model-informed drug development. CPT Pharmacometrics Syst Pharmacol. 2019;8(1):9-11. https://doi.org/10.1002/psp4.12379.
Bonate PL. What happened to the modeling and simulation revolution? Clin Pharmacol Ther. 2014;96(4):416-417.
Bonate P. Be a Model Communicator: And Sell Your Models to Anyone. ISBN-10: 0692323813, ISBN-13: 978-0692323816, 2014.
Musuamba FT, Bursi R, Manolis E, et al. Verifying and validating quantitative systems pharmacology and In silico models in drug development: current needs, gaps, and challenges. CPT Pharmacometrics Syst. Pharmacol. 2020;9:195-197.
https://www.oecd.org/chemicalsafety/risk-assessment/oecdquantitativestructure-activityrelationshipsprojectqsars.htm. Accessed January 24, 2021.
EMEA. Guideline on the Limits of Genotoxic Impurities. Committee for Medicinal Products for Human Use (CPMP). European Medicines Agency. 2006. London, 28 June 2006, CPMP/SWP/5199/02. EMEA/CHMP/QWP/251344/2006. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500002903.pdf. Accessed January 24, 2021.
Committee for Medicinal Products for Human Use (CHMP). Guideline on reporting the results of population pharmacokinetic analysis. https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-reporting-results-population-pharmacokinetic-analyses_en.pdf. Accessed January 24, 2021.
Musuamba F, Manolis E, Holford N, et al. Advanced methods for dose and regimen finding during drug development: summary of the EMA/EFPIA workshop on dose finding (London 4–5 December 2014). CPT Pharmacometrics Syst. Pharmacol. 2017;6:418-429. https://doi.org/10.1002/psp4.12196.
European Medicines Agency (EMA). Reflection paper on the use of extrapolation in the development of medicines for paediatrics. https://www.ema.europa.eu/en/documents/scientific-guideline/adopted-reflection-paper-use-extrapolation-development-medicines-paediatrics-revision-1en.pdf. Accessed January 24, 2021.
European Medicines Agency (EMA). Guideline on the reporting of physiologically based pharmacokinetic (PBPK) modelling and simulation. https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-reporting-physiologically-based-pharmacokinetic-pbpk-modelling-simulation_en.pdf. Accessed January 24, 2021.
US Food and Drug Administration (FDA). Physiologically based pharmacokinetic analyses — format and content guidance for industry. https://www.fda.gov/media/101469/download. Accessed January 24, 2021.
The American Society of Mechanical Engineers (ASME). Assessing credibility of computational modeling through verification and validation: application to medical devices VV40 – 2018. https://www.asme.org/products/codes-standards/v-v-40-2018-assessing-credibility-computational. Accessed January 24, 2021.
https://www.ema.europa.eu/en/human-regulatory/research-development/scientific-advice-protocol-assistance/qualification-novel-methodologies-medicine-development-0. Accessed January 24, 2021.
Systems biology Ontology. http://www.ebi.ac.uk/sbo/main/. accessed 12/5/2021
Geris L. Regenerative orthopaedics: In vitro, in vivo … in silico. Int Orthop. 2014;38:1771-1778.
Kohl P, Noble D. Systems biology and the virtual physiological human. Mol Syst Biol. 2020;5:292.
Stéphanou A, Fanchon E, Innominato PF, Ballesta A. Systems biology, systems medicine, systems pharmacology: The what and the why. Acta Biotheor. 2018;66:345-365.
Knight-Schrijver VR, Chelliah V, Cucurull-Sanchez L, Le Novère N. The promises of quantitative systems pharmacology modelling for drug development. Comput. Struct. Biotechnol. J. 2016;14:363-370.
Chabaud S, Girard P, Nony P. Boissel JP on behalf of the THERMOS group. Clinical trial simulation using therapeutic effect modelling: application to ivabradine efficacy in patients with angina pectoris. J Pharmacokinetics Pharmacodynamics. 2002;29:339-363.
Ribba B, Saut O, Colin T, Bresch D, Grenier E, Boissel JP. A multiscale mathematical model of avascular tumor growth to investigate the therapeutic benefit of anti-invasive agents. J Theor Biol. 2006;243:532-534.
Billy F, Ribba B, Saut O, et al. A pharmacologically based multiscale mathematical model of angiogenesis and its use in investigating the efficacy of a new cancer treatment strategy. J Theor Biol. 2009;260:545-562.
Barrett JS, Fossler MJ, Cadieu KD, Gastonguay MR. Pharmacometrics: A multidisciplinary field to facilitate critical thinking in drug development and translational research settings. The Journal of Clinical Pharmacology. 2008;48(5):632-649. https://doi.org/10.1177/0091270008315318.
Viceconti M, Henney A, Morley-Fletcher E. In silico clinical trials: how computer simulation will transform the biomedical industry. Int. J. Clin. Trials. 2016;3:37.
ISW, GSP Task Force - In silico World. http://insilico.world/practice/good-simulation-practices/gsp-task-force/#what. Accessed January 24, 2021.
Viceconti M, Pappalardo F, Rodriguez B, Horner M, Bischoff J, Musuamba Tshinanu F. In silico trials: Verification, validation and uncertainty quantification of predictive models used in the regulatory evaluation of biomedical products. Methods. 2021;2023(19):120-127. https://doi.org/10.1016/j.ymeth.2020.01.011.
AI definition. https://ec.europa.eu/digital-single-market/en/news/definition-artificial-intelligence-main-capabilities-and-scientific-disciplines. Accessed January 24, 2021.
Williams PJ, Ette EI. Pharmacometrics: impacting drug development and pharmacotherapy. In: Ette EI, Williams PJ, eds. Pharmacometrics: The Science of Quantitative Pharmacology. Hoboken, NJ: John Wiley & Sons; 2007:1-21.
Mould DR, Upton RN. Basic concepts in population modeling, simulation, and model-based drug development-part 2: introduction to pharmacokinetic modeling methods. CPT: Pharmacometrics & Systems Pharmacology. 2013;2(4):38. https://doi.org/10.1038/psp.2013.14.
Derendorf H, Meibohm B. Modeling of pharmacokinetic/pharmacodynamic (PK/PD) relationships: concepts and perspectives. Pharm Res. 1999;16:176-185.
Apweiler R, Beissbarth T, Berthold M, et al. Whither systems medicine? Exp Mol Med. 2018;50(3):e453.
https://www.ima.umn.edu/materials/2017-2018/SW3.6-8.18/26864/Himes-IMA-2018-MDIC_virtual_patients.pdf. Accessed January 24, 2021.
ISW. Good Simulation Practices - In silico World. https://insilico.world/practice/good-simulation-practices/. Accessed January 24, 2021.
Loizou G, Spendiff M, Barton HA, et al. Development of good modelling practice for physiologically based pharmacokinetic models for use in risk assessment: The first steps. Regul Toxicol Pharmacol. 2008;50:400-411.
Nguyen TH, Mouksassi MS, Holford N, et al. Model Evaluation of Continuous Data Pharmacometric Models: Metrics and Graphics. CPT: Pharmacometrics & Systems Pharmacology. 2016;6(2):87-109. https://doi.org/10.1002/psp4.12161.
https://www.mobihealthnews.com/content/eli-lilly-gets-fda-clearance-insulin-dose-calculator-app. Accessed January 24, 2021.
https://www.ema.europa.eu/en/human-regulatory/research-development/innovation-medicines#ema's-innovation-task-force-(itf)-section. Accessed January 24, 2021.
https://www.ema.europa.eu/en/human-regulatory/research-development/scientific-advice-protocol-assistance. Accessed January 24, 2021.
https://www.ema.europa.eu/en/human-regulatory/research-development/scientific-advice-protocol-assistance/qualification-novel-methodologies-medicine-development-0. Accessed January 24, 2021.
Madabuchi R, Benjamin JM, Grewal R, et al. The US food and drug administration's model-informed drug development paired meeting pilot program: early experience and impact. Clin Pharmacol Ther. 2019;106(1):74-78. https://doi.org/10.1002/cpt.1457.
https://www.ema.europa.eu/en/human-regulatory/research-development/scientific-guidelines/clinical-pharmacology-pharmacokinetics/modelling-simulation-questions-answers. Accessed January 24, 2021.
https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-use-pharmacokinetics-pharmacodynamics-development-antimicrobial-medicinal-products_en.pdf. Accessed January 24, 2021.
https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/guidance-companies-considering-adaptive-pathways-approach_en.pdf. Accessed January 24, 2021.
https://www.ema.europa.eu/en/documents/report/european-federation-pharmaceutical-inudustries-associations-european-medicines-agency-modelling_en.pdf. Accessed January 24, 2021.
Manolis E, Rohou S, Hemmings R, Salmonson T, Karlsson M, Milligan PA. The role of modeling and simulation in development and registration of medicinal products: output from the EFPIA/EMA modeling and simulation workshop. CPT Pharmacometrics Syst Pharmacol. 2013;2(2):e31. https://doi.org/10.1038/psp.2013.7.
http://www.r-project.org/doc/R-FDA.pdf. Accessed January 24, 2021.
Hariharan P, D'Souza G, Horner M, Malinauskas RA, Myers MR. Verification benchmarks to assess the implementation of computational fluid dynamics based hemolysis prediction models. J Biomech Eng. 2015;137(9). https://doi.org/10.1115/1.4030823.
Bhattacharya P, Viceconti M. Multiscale modeling methods in biomechanics. Wiley Interdisciplinary Rev. Syst. Biol. Med. 2017;9:e1375.
Viceconti M, Juárez MA, Curreli C, Pennisi M, Russo G, Pappalardo F. Credibility of in silico trial technologies—a theoretical framing. IEEE Journal of Biomedical and Health Informatics. 2020;24(1):4-13. https://doi.org/10.1109/JBHI.2019.2949888.
Hengl S, Kreutz D, Timmer J, Maiwald T. Data-based identifiability analysis of non-linear dynamical models. Bioinformatics. 2007;23(19):2612-2618.
Raue A, Kreutz C, Maiwald T. Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood. Author Notes Bioinformatics. 2009;25(15):1923-1929.
Erdemir A, Mulugeta L, Ku JP, et al. Credible practice of modeling and simulation in healthcare: ten rules from a multidisciplinary perspective. J Transl Med. 2020;18:369. https://doi.org/10.1186/s12967-020-02540-4.
Klipp E, et al. Standards in computational systems biology 2006. https://www.ebi.ac.uk/compneur-srv/biomodels/doc/KlippSurvey.pdf. accessed 12/5/2021;
Alvarez-Builla R, et al., Modeling methods for medical systems biology, 2018.
Tiwari K, Kananathan S, Roberts MG, et al. Reproducibility in systems biology modelling. Mol Syst Biol. 2021;17:e9982. https://doi.org/10.15252/msb.20209982.
Novère N, Finney A, Hucka M, et al. Minimum information requested in the annotation of biochemical models (MIRIAM). Nat Biotechnol. 2005;23:1509-1515. https://doi.org/10.1038/nbt1156.
Klipp E, Liebermeister W, Helbig A, et al. Systems biology standards—the community speaks. Nat Biotechnol. 2007;25:390-391. https://doi.org/10.1038/nbt0407-390.
Foundation for Research on Information Technologies in Society. https://itis.swiss/virtual-population/tissue-properties/overview/. Accessed January 24, 2021.
Kuemmel C, Yang Y, Zhang X, et al. Consideration of a credibility assessment framework in model-informed drug development: potential application to physiologically-based pharmacokinetic modeling and simulation. CPT Pharmacometrics Syst. Pharmacol. 2020;9:21-28.
Pathmanathan P, Cordeiro JM, Gray RA. Comprehensive uncertainty quantification and sensitivity analysis for cardiac action potential models. Front Physiol. 2019;26;10:721. https://doi.org/10.3389/fphys.2019.00721. eCollection 2019.
Grossfield A, Patrone PN, Roe DR, Schultz AJ, Siderius D, Zuckerman DM. Best practices for quantification of uncertainty and sampling quality in molecular simulations [Article v1.0]. Living J Com Mol Sci. 2019;1(1). https://doi.org/10.33011/livecoms.1.1.5067.
Pappalardo F, Russo G, Pennisi M, et al. The potential of computational modeling to predict disease course and treatment response in patients with relapsing multiple sclerosis. Cells. 2020;9(3):586. https://doi.org/10.3390/cells9030586.
Gianì F, Russo G, Pennisi M, Sciacca L, Frasca F, Pappalardo F. Computational modeling reveals MAP3K8 as mediator of resistance to vemurafenib in thyroid cancer stem cells. Bioinformatics. 2019;35(13):2267-2275. https://doi.org/10.1093/bioinformatics/bty969.
Pennisi M, Russo G, Ravalli S, Pappalardo F. Combining agent based-models and virtual screening techniques to predict the best citrus-derived vaccine adjuvants against human papilloma virus. BMC Bioinformatics. 2017;18:544. https://doi.org/10.1186/s12859-017-1961-9.
Palladini A, Nicoletti G, Pappalardo F, et al. In silico modelling and in vivo efficacy of cancer-preventive vaccinations. Cancer Res. 2010;70:7755-7763. https://doi.org/10.1158/0008-5472.CAN-10-0701.
Gullo F, van der Garde M, Russo G, et al. Computational modeling of the expansion of human cord blood CD133+ hematopoietic stem/progenitor cells with different cytokine combinations. Bioinformatics. 2015;31(15):2514-2522. https://doi.org/10.1093/bioinformatics/btv172.
Pappalardo F, Fichera E, Paparone N, et al. A computational model to predict the immune system activation by citrus derived vaccine adjuvants. Bioinformatics. 2016;32(17):2672-2680.
Pennisi M, Russo G, Sgroi G, et al. Predicting the artificial immunity induced by RUTI vaccine against tuberculosis using universal immune system simulator (UISS). BMC Bioinformatics. 2019;20(S6):504. https://doi.org/10.1186/s12859-019-3045-5.
Russo G, Sgroi G, Parasiliti Palumbo GA, et al. Moving forward through the in silico modeling of tuberculosis: a further step with UISS-TB. BMC Bioinformatics. 2020;21(S17):458. https://doi.org/10.1186/s12859-020-03762-5.
Juarez MA, Ennisi GR, Dimitrios K, Cristina C, Marco V, Francesco P. Generation of digital patients for the simulation of tuberculosis with UISS-TB. BMC Bioinformatics 2020;21(S17):449. https://doi.org/10.1186/s12859-020-03776-z.
Russo G, Pappalardo F, Juarez MA, et al. Evaluation of the efficacy of RUTI and ID93/GLA-SE vaccines in tuberculosis treatment: in silico trial through UISS-TB simulator. In 3rd International Workshop on Computational Methods for the Immune System Function (CMISF 2019), IEEE International Conference on Bioinformatics and Biomedicine (BIBM 2019), 2019:2197–2201. https://doi.org/10.1109/BIBM47256.2019.8983060
Passini E, Britton OJ, Bueno-Orovio A, Rodriguez B. Human in silico trials on drug-induced changes in electrophysiology and calcium dynamics using the virtual assay software. J Pharmacol Toxicol Methods. 2019;99:106595. https://doi.org/10.1016/j.vascn.2019.05.104.
Passini E, Britton OJ, Lu HR, et al. Human in silico drug trials demonstrate higher accuracy than animal models in predicting clinical pro-arrhythmic cardiotoxicity. Front Physiol. 2017;8:668. https://doi.org/10.3389/fphys.2017.00668.
Zhou X, Qu Y, Passini E, et al. Blinded in silico drug trial reveals the minimum set of ion channels for torsades de pointes risk assessment. Frontiers in Pharmacology. 2020;10:1643. https://doi.org/10.3389/fphar.2019.01643.
Passini E, Trovato C, Morissette P, Sannajust F, Bueno-Orovio A, Rodriguez B. Drug-induced shortening of the electromechanical window is an effective biomarker for in silico prediction of clinical risk of arrhythmias. Br J Pharmacol. 2019;176(19):3819-3833. https://doi.org/10.1111/bph.14786.
