[en] Crop simulation models are important tools in agronomy. Typically, they need to be calibrated before being used for new environments or cultivars. However, there is a large variability in calibration approaches, which contributes to uncertainty in simulated values, so it is important to develop improved calibration procedures that are widely applicable. The AgMIP calibration group recently proposed a comprehensive, generic calibration protocol that is directly based on standard statistical parameter estimation in regression models. Weighted least squares (WLS) is used to handle multiple response variables and forward regression using the corrected Akaike Information Criterion (AICc) is used to select the parameters to be calibrated. The protocol includes two adaptations, which are specific to each model and data set. First, initial approximations to the WLS parameters are obtained by fitting variables one group at a time. Secondly, “major” parameters are identified that are intended to reduce bias, analogously to the constant in linear regression. In this study, new diagnostic tools to be included in the protocol are proposed and tested in a case study. The diagnostics test whether the protocol does indeed lead to good initial approximations to the WLS parameters, and whether the protocol does indeed substantially reduce bias. These diagnostics provide in-depth understanding of the calibration process, reveal problems and help suggest solutions. The diagnostics should increase confidence in the results of the protocol. Having a reliable, generic calibration approach, like the augmented AgMIP protocol, is essential to using crop models more effectively.
Disciplines :
Agriculture & agronomy Computer science
Author, co-author :
Wallach, Daniel ; Institute of Crop Science and Resource Conservation, University of Bonn, Bonn, Germany
Kim, Kwang Soo; Department of Agriculture, Forestry, and Bioresources, Seoul National University, South Korea
Hyun, Shinwoo ; Department of Agriculture, Forestry, and Bioresources, Seoul National University, South Korea
Buis, Samuel; INRAE, UMR 1114 EMMAH, Avignon, France
Thorburn, Peter ; CSIRO Agriculture and Food, Brisbane, Australia
Mielenz, Henrike ; Julius Kühn Institute (JKI) – Federal Research Centre for Cultivated Plants, Institute for Crop and Soil Science, Braunschweig, Germany
Seidel, Sabine Julia ; Institute of Crop Science and Resource Conservation, University of Bonn, Bonn, Germany ; Institute of Organic Farming, Department of Agricultural Sciences, University of Natural Resources and Life Sciences, Vienna, Austria
Alderman, Phillip D. ; Department of Plant and Soil Sciences, Oklahoma State University, Stillwater, United States
Dumont, Benjamin ; Université de Liège - ULiège > TERRA Research Centre > Plant Sciences
Fallah, Mohammad Hassan ; Department of Agrotechnology, Faculty of Agriculture, Ferdowsi University of Mashhad, Mashhad, Iran
Hoogenboom, Gerrit; Agricultural and Biological Engineering Department, University of Florida, Gainesville, United States ; Global Food Systems Institute, University of Florida, Gainesville, United States
Justes, Eric ; CIRAD, Persyst Department, Montpellier, France
Kersebaum, Kurt-Christian ; Leibniz Centre for Agricultural Landscape Research (ZALF), Müncheberg, Germany ; Global Change Research Institute CAS, Brno, Czech Republic ; Tropical Plant Production and Agricultural Systems Modelling (TROPAGS), Dep.of Crop Sciences, Georg-August-University of Göttingen, Göttingen, Germany
Launay, Marie; INRAE, US 1116 AgroClim, Avignon, France
Leolini, Luisa; Department of Agriculture, Food, Environment and Forestry (DAGRI), University of Florence, Florence, Italy
Mehmood, Muhammad Zeeshan ; Department of Plant and Soil Sciences, Oklahoma State University, Stillwater, United States
Moriondo, Marco; CNR-IBE, Firenze, Italy
Jing, Qi; Ottawa Research and Development Centre, Agriculture and Agri-Food Canada, Ottawa, Canada
Qian, Budong; Ottawa Research and Development Centre, Agriculture and Agri-Food Canada, Ottawa, Canada
Susanne, Schulz; Leibniz Centre for Agricultural Landscape Research (ZALF), Müncheberg, Germany
Seserman, Diana-Maria; Leibniz Centre for Agricultural Landscape Research (ZALF), Müncheberg, Germany
Shelia, Vakhtang; Agricultural and Biological Engineering Department, University of Florida, Gainesville, United States ; Global Food Systems Institute, University of Florida, Gainesville, United States
Weihermüller, Lutz; Institute of Bio, and Geosciences - IBG-3, Agrosphere, Forschungszentrum Jülich GmbH, Jülich, Germany
Palosuo, Taru ; Natural Resources Institute Finland (Luke), Helsinki, Finland
This work has partially been funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany\u2019s Excellence Strategy - EXC 2070 \u2013 390732324 , by funds of the Federal Ministry of Food and Agriculture (BMEL) based on a decision of the Parliament of the Federal Republic of Germany via the Federal Office for Agriculture and Food (BLE), grant number 2822ABS010 , the European Union (EU horizon project IntercropVALUES, grant agreement No 101081973), the Federal Ministry of Education and Research (BMBF) BonaRes Project I4S (grant Number 031B1069B), the Ministry of Culture and Science of the German State of North Rhine-Westphalia (MKW) under the Excellence Strategy of the Federal and State Governments, and Rural Development Administration (RDA), Republic of Korea under the Cooperative Research Program for Agriculture Science & Technology Development (Project No. RS-2024-00361442 ), the Ministry of Education, Youth and Sports of the Czech Republic (grant AdAgriF - Advanced methods of greenhouse gases emission reduction and sequestration in agriculture and forest landscape for climate change mitigation ( CZ.02.01.01/00/22_008/0004635 ), the INRAE CLIMAE meta-program and AgroEcoSystem department.
Aho, K., Derryberry, D., Peterson, T., Model selection for ecologists: the worldviews of AIC and BIC. Ecology 95 (2014), 631–636, 10.1890/13-1452.1.
Ahuja, L.R., Ma, L., A synthesis of current parameterization approaches and needs for further improvements, in: methods of introducing system models into agricultural research. Adv. Agric. Syst. Model., 2011, 427–440, 10.2134/advagricsystmodel2.c15.
ARVALIS, 2022. Les températures chaudes en mai accélèrent la maturité des céréales d'hiver [WWW Document]. Infos Tech. URL 〈https://www.arvalis.fr/infos-techniques/les-temperatures-chaudes-en-mai-accelerent-la-maturite-des-cereales-dhiver〉 (accessed 10.29.24).
Berton Ferreira, T., Shelia, V., Porter, C., Moreno Cadena, P., Salmeron Cortasa, M., Sohail Khan, M., Pavan, W., Hoogenboom, G., Enhancing crop model parameter estimation across computing environments: utilizing the GLUE method and parallel computing for determining genetic coefficients. Comput. Electron. Agric., 227, 2024, 109513, 10.1016/J.COMPAG.2024.109513.
Boote, K., (eds.) Advances in crop modelling for a sustainable agriculture, 2019, Burleigh Dodds Science Publishing 〈https://doi.org/10.1201/9780429266591〉.
Boote, K.J., Jones, J.W., Hoogenboom, G., White, J.W., The role of crop systems simulation in agriculture and environment. Int. J. Agric. Environ. Inf. Syst. 1 (2010), 41–54.
Buis, S., Lecharpentier, P., Vezy, R., Ginet, M., 2023. CroptimizR: A Package to Estimate Parameters of Crop Models [WWW Document]. URL https://doi.org/10.5281/zenodo.4066451 (2023).
César Trejo Zúñiga, E., López Cruz, I.L., García, A.R., Parameter estimation for crop growth model using evolutionary and bio-inspired algorithms. Appl. Soft Comput. 23 (2014), 474–482, 10.1016/J.ASOC.2014.06.023.
Claeskens, G., Hjort, N.L., Model Selection and Model Averaging. 2001, Cambridge University Press 〈https://doi.org/10.1017/CBO9780511790485〉.
Dua, V.K., Minhas, J.S., Rawal, S., Singh, S.P., Singh, S.K., KUMAR, P., PATHANIA, R., Kapoor, T., SHARMA, J., Sharma, S.K., Mankar, P., Kawat, S., Singh, B.P., Chakrabarti, S.K., Calibration and validation of WOFOST model for seven potato (Solanum tuberosum) cultivars in India. Indian J. Agron. 63 (2018), 357–365, 10.59797/IJA.V63I3.5661.
Fallah, M.H., Nezami, A., Khazaie, H.R., Nassiri Mahallati, M., Evaluation of DSSAT-Nwheat model across a wide range of climate conditions in Iran. J. Agroecol 12 (2020), 561–580, 10.22067/jag.v12i4.77250.
Gao, Y., Wallach, D., Hasegawa, T., Tang, L., Zhang, R., Asseng, S., Kahveci, T., Liu, L., He, J., Hoogenboom, G., Evaluation of crop model prediction and uncertainty using Bayesian parameter estimation and Bayesian model averaging. Agric. For. Meteor., 311, 2021, 108686, 10.1016/J.AGRFORMET.2021.108686.
Grassini, P., van Bussel, L.G.J., Van Wart, J., Wolf, J., Claessens, L., Yang, H., Boogaard, H., de Groot, H., van Ittersum, M.K., Cassman, K.G., How good is good enough? Data requirements for reliable crop yield simulations and yield-gap analysis. F. Crop. Res. 177 (2015), 49–63, 10.1016/j.fcr.2015.03.004.
Guillaume, S., Berez, J.-E., Wallach, D., Justes, E., Methodological comparison of calibration procedures for durum wheat parameters in the STICS model. Eur. J. Agron. 35 (2011), 115–126.
Guo, D., Olesen, J.E., Pullens, J.W.M., Guo, C., Ma, X., Calibrating AquaCrop model using genetic algorithm with multi-objective functions applying different weight factors. Agron. J. 113 (2021), 1420–1438, 10.1002/agj2.20588.
Hargreaves, J.C., Skill and uncertainty in climate models. WIREs Clim. Chang 1 (2010), 556–564, 10.1002/wcc.58.
Harrison, M.T., Roggero, P.P., Zavattaro, L., Simple, efficient and robust techniques for automatic multi-objective function parameterisation: Case studies of local and global optimisation using APSIM. Environ. Model. Softw. 117 (2019), 109–133, 10.1016/j.envsoft.2019.03.010.
He, D., Wang, E., Wang, J., Robertson, M.J., Data requirement for effective calibration of process-based crop models. Agric. For. Meteor., 2017, 10.1016/j.agrformet.2016.12.015.
Hlavinka, P., Trnka, M., Kersebaum, K., čermák, P., Pohanková, E., Orság, M., Pokorný, E., Fischer, M., Brtnický, M., žalud, Z., Modelling of yields and soil nitrogen dynamics for crop rotations by HERMES under different climate and soil conditions in the Czech Republic. J. Agric. Sci. 152 (2013), 188–204, 10.1017/S0021859612001001.
Hoogenboom, G., Porter, C.H., Boote, K.J., Shelia, V., Wilkens, P.W., Singh, U., White, J.W., Asseng, S., Lizaso, J.I., Moreno, L.P., Pavan, W., Ogoshi, R., Hunt, L.A., Tsuji, G.Y., Jones, J.W., The DSSAT crop modeling ecosystem, in: Boote, K. J. (Ed.), Advances in Crop Modeling for a Sustainable Agriculture. Burleigh Dodds Sci. Publ., Camb. U. Kingd., 2019, 173–216, 10.19103/AS.2019.0061.10.
Jha, P.K., Ines, A.V.M., Singh, M.P., A multiple and ensembling approach for calibration and evaluation of genetic coefficients of CERES-Maize to simulate maize phenology and yield in Michigan. Environ. Model. Softw., 135, 2021, 104901, 10.1016/j.envsoft.2020.104901.
Jha, P.K., Ines, A.V.M., Han, E., Cruz, R., Vara Prasad, P.V., A comparison of multiple calibration and ensembling methods for estimating genetic coefficients of CERES-Rice to simulate phenology and yields. F. Crop. Res., 284, 2022, 108560, 10.1016/J.FCR.2022.108560.
Jing, Q., McConkey, B., Qian, B., Smith, W., Grant, B., Shang, J., Liu, J., Bindraban, P., Luce, M., St, Assessing water management effects on spring wheat yield in the Canadian Prairies using DSSAT wheat models. Agric. Water Manag., 244, 2021, 106591, 10.1016/J.AGWAT.2020.106591.
Kassie, B.T., Asseng, S., Porter, C.H., Royce, F.S., Performance of DSSAT-Nwheat across a wide range of current and future growing conditions. Eur. J. Agron. 81 (2016), 27–36, 10.1016/J.EJA.2016.08.012.
Kersebaum, K.C., Special Features of the HERMES Model and Additional Procedures for Parameterization, Calibration, Validation, and Applications. Ahuja, L.R., Ma, L., (eds.) Methods of Introducing System Models into Agricultural Research, 2011, John Wiley & Sons, Ltd, 65–94 〈https://doi.org/10.2134/advagricsystmodel2.c2〉.
Kobayashi, K., Salam, M.U., Comparing simulated and measured values using mean squared deviation and its components. Agron. J. 92 (2000), 345–352.
Kumar, K., 2023. Exploring Optimization Techniques for Parameter Estimation in Nonlinear System Modeling.
Liu, L., Wallach, D., Li, J., Liu, B., Zhang, L., Tang, L., Zhang, Y., Qiu, X., Cao, W., Zhu, Y., Uncertainty in wheat phenology simulation induced by cultivar parameterization under climate warming. Eur. J. Agron. 94 (2018), 46–53, 10.1016/J.EJA.2017.12.001.
Morozova, O., Levina, O., Uusküla, A., Heimer, R., Comparison of subset selection methods in linear regression in the context of health-related quality of life and substance abuse in Russia. BMC Med. Res. Method., 15, 2015, 71, 10.1186/s12874-015-0066-2.
Pasley, H., Brown, H., Holzworth, D., Whish, J., Bell, L., Huth, N., How to build a crop model. A review. Agron. Sustain. Dev., 43, 2023, 2, 10.1007/s13593-022-00854-9.
Richter, G.M., Acutis, M., Trevisiol, P., Latiri, K., Confalonieri, R., Sensitivity analysis for a complex crop model applied to Durum wheat in the Mediterranean. Eur. J. Agron. 32 (2010), 127–136, 10.1016/j.eja.2009.09.002.
Seber, G.A.F., Wild, C.J., Nonlinear regression. 1989, Wiley, New York.
Seidel, S.J., Palosuo, T., Thorburn, P., Wallach, D., Towards improved calibration of crop models – Where are we now and where should we go?. Eur. J. Agron. 94 (2018), 25–35, 10.1016/J.EJA.2018.01.006.
Turgal, E., Doganay, B., Incl. Exclud. a Constant Term. Regres. Anal., 2017.
Van Oijen, M., Rougier, J., Smith, R., Bayesian calibration of process-based forest models: bridging the gap between models and data. Tree Physiol. 25 (2005), 915–927.
Vezy, R., Buis, S., Lecharpentier, P., Giner, M., 2023. CroPlotR: a package to analyse crop model simulations outputs with plots and statistics. WWW Doc., 2023, 10.5281/zenodo.4066451.
Wallach, D., Crop model calibration: a statistical perspective. Agron. J. 103 (2011), 1144–1151.
Wallach, D., Palosuo, T., Thorburn, P., Gourdain, E., Asseng, S., Basso, B., Buis, S., Crout, N., Dibari, C., Dumont, B., Ferrise, R., Gaiser, T., Garcia, C., Gayler, S., Ghahramani, A., Hochman, Z., Hoek, S., Hoogenboom, G., Horan, H., Huang, M., Jabloun, M., Jing, Q., Justes, E., Kersebaum, K.C., Klosterhalfen, A., Launay, M., Luo, Q., Maestrini, B., Mielenz, H., Moriondo, M., Nariman Zadeh, H., Olesen, J.E., Poyda, A., Priesack, E., Pullens, J.W.M., Qian, B., Schütze, N., Shelia, V., Souissi, A., Specka, X., Srivastava, A.K., Stella, T., Streck, T., Trombi, G., Wallor, E., Wang, J., Weber, T.K.D., Weihermüller, L., de Wit, A., Wöhling, T., Xiao, L., Zhao, C., Zhu, Y., Seidel, S.J., How well do crop modeling groups predict wheat phenology, given calibration data from the target population?. Eur. J. Agron., 124, 2021, 126195, 10.1016/j.eja.2020.126195.
Wallach, D., Palosuo, T., Thorburn, P., Hochman, Z., Gourdain, E., Andrianasolo, F., Asseng, S., Basso, B., Buis, S., Crout, N., Dibari, C., Dumont, B., Ferrise, R., Gaiser, T., Garcia, C., Gayler, S., Ghahramani, A., Hiremath, S., Hoek, S., Horan, H., Hoogenboom, G., Huang, M., Jabloun, M., Jansson, P.-E., Jing, Q., Justes, E., Kersebaum, K.C., Klosterhalfen, A., Launay, M., Lewan, E., Luo, Q., Maestrini, B., Mielenz, H., Moriondo, M., Nariman Zadeh, H., Padovan, G., Olesen, J.E., Poyda, A., Priesack, E., Pullens, J.W.M., Qian, B., Schütze, N., Shelia, V., Souissi, A., Specka, X., Srivastava, A.K., Stella, T., Streck, T., Trombi, G., Wallor, E., Wang, J., Weber, T.K.D., Weihermüller, L., de Wit, A., Wöhling, T., Xiao, L., Zhao, C., Zhu, Y., Seidel, S.J., The chaos in calibrating crop models: Lessons learned from a multi-model calibration exercise. Environ. Model. Softw., 145, 2021, 105206, 10.1016/J.ENVSOFT.2021.105206.
Wallach, D., Palosuo, T., Thorburn, P., Mielenz, H., Buis, S., Hochman, Z., Gourdain, E., Andrianasolo, F., Dumont, B., Ferrise, R., Gaiser, T., Garcia, C., Gayler, S., Harrison, M., Hiremath, S., Horan, H., Hoogenboom, G., Jansson, P.-E., Jing, Q., Justes, E., Kersebaum, K.-C., Launay, M., Lewan, E., Liu, K., Mequanint, F., Moriondo, M., Nendel, C., Padovan, G., Qian, B., Schütze, N., Seserman, D.-M., Shelia, V., Souissi, A., Specka, X., Srivastava, A.K., Trombi, G., Weber, T.K.D., Weihermüller, L., Wöhling, T., Seidel, S.J., Proposal and extensive test of a calibration protocol for crop phenology models. Agron. Sustain. Dev., 43, 2023, 46, 10.1007/s13593-023-00900-0.
Wallach, D., Buis, S., Seserman, D.-M., Palosuo, T., Thorburn, P.J., Mielenz, H., Justes, E., Kersebaum, K.-C., Dumont, B., Launay, M., Seidel, S.J., A calibration protocol for soil-crop models. Environ. Model. Softw., 180, 2024, 106147, 10.1016/J.ENVSOFT.2024.106147.
Wang, X., Jeong, J., Park, S., Zhang, X., Gao, J., Silvero, N., DayCent-CUTE: a global sensitivity, auto-calibration, and uncertainty analysis tool for DayCent. Environ. Model. Softw., 2023, 105832, 10.1016/J.ENVSOFT.2023.105832.
Winn, C.A., Archontoulis, S., Edwards, J., Calibration of a crop growth model in APSIM for 15 publicly available corn hybrids in North America. Crop Sci. 63 (2023), 511–534, 10.1002/csc2.20857.
Wolf, J., Evans, L.G., Semenov, M.A., Eckersten, H., Iglesias, A., Comparison of wheat simulation models under climate change. I. Model calibration and sensitivity analyses. Clim. Res. 7 (1996), 253–270.
Zhao, G., Bryan, B.A., Song, X., Sensitivity and uncertainty analysis of the APSIM-wheat model: Interactions between cultivar, environmental, and management parameters. Ecol. Model. 279 (2014), 1–11, 10.1016/j.ecolmodel.2014.02.003.
