[en] Process-based soil-crop models are widely used in agronomic research. They are major tools for evaluating climate change impact on crop production. Multi-model simulation studies show a wide diversity of results among models, implying that simulation results are very uncertain. A major path to improving simulation results is to propose improved calibration practices that are widely applicable. This study proposes an innovative generic calibration protocol. The two major innovations concern the treatment of multiple output variables and the choice of parameters to estimate, both of which are based on standard statistical procedure adapted to the particularities of soil-crop models. The protocol performed well in a challenging artificial-data test. The protocol is formulated so as to be applicable to a wide range of models and data sets. If widely adopted, it could substantially reduce model error and inter-model variability, and thus increase confidence in soil-crop model simulations.
Disciplines :
Agriculture & agronomy Computer science
Author, co-author :
Wallach, Daniel ; Institute of Crop Science and Resource Conservation, University of Bonn, Bonn, Germany
Buis, Samuel; INRAE, UMR 1114 EMMAH, Avignon, France
Seserman, Diana-Maria; Leibniz Centre for Agricultural Landscape Research (ZALF), Müncheberg, Germany
Palosuo, Taru ; Natural Resources Institute Finland (Luke), Helsinki, Finland
Thorburn, Peter J.; CSIRO Agriculture and Food, Brisbane, Australia
Mielenz, Henrike; Julius Kühn Institute (JKI) – Federal Research Centre for Cultivated Plants, Institute for Crop and Soil Science, Braunschweig, Germany
Justes, Eric; CIRAD, Persyst Department, Montpellier, France
Kersebaum, Kurt-Christian; Leibniz Centre for Agricultural Landscape Research (ZALF), Müncheberg, Germany ; Global Change Research Institute CAS, Brno, Czech Republic ; Tropical Plant Production and Agricultural Systems Modelling (TROPAGS), Georg-August-University Göttingen, Göttingen, Germany
Dumont, Benjamin ; Université de Liège - ULiège > TERRA Research Centre > Plant Sciences
Launay, Marie; INRAE, US 1116 AgroClim, Avignon, France
Seidel, Sabine Julia ; Institute of Crop Science and Resource Conservation, University of Bonn, Bonn, Germany
DFG - Deutsche Forschungsgemeinschaft BMBF - Federal Ministry of Education and Research
Funding text :
This study was carried out in the framework of the Agricultural Model Intercomparison and Improvement Project (AgMIP). The presented study has been funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy-EXC 2070-390732324 (Phenorob) and by the Federal Ministry of Education and Research (BMBF) and the Ministry of Culture and Science of the German State of North Rhine-Westphalia (MKW) under the Excellence Strategy of the Federal and State Governments. KCK was supported by AdAgriF - Advanced methods of greenhouse gases emission reduction and sequestration in agriculture and forest landscape for climate change mitigation (CZ.02.01.01/00/22_008/0004635). Partial funding was provided by the BonaRes project Soil3 (BOMA 03037514, 031B0515C) of the Federal Ministry of Education and Research (BMBF), Germany and the INRAE CLIMAE meta-program and AgroEcoSystem department. The presented study has been also cofunded by the European Union (EU Horizon project IntercropVALUES, grant agreement No 101081973).This study was carried out in the framework of the Agricultural Model Intercomparison and Improvement Project (AgMIP). KCK was supported by AdAgriF - Advanced methods of greenhouse gases emission reduction and sequestration in agriculture and forest landscape for climate change mitigation (CZ.02.01.01/00/22_008/0004635). Partial funding was provided by the BonaRes project Soil3 (BOMA 03037514, 031B0515C) of the Federal Ministry of Education and Research (BMBF), Germany, and by Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany\u2019s Excellence Strategy-EXC 2070 \u2013 390732324.
Aho, K., Derryberry, D., Peterson, T., Model selection for ecologists: the worldviews of AIC and BIC. Ecology 95 (2014), 631–636, 10.1890/13-1452.1.
Ahuja, L.R., Ma, L., (eds.) Methods of Introducing System Models into Agricultural Research, Advances in Agricultural Systems Modeling, 2011, American Society of Agronomy and Soil Science Society of America, Madison, WI, USA, 10.2134/advagricsystmodel2.
Albanito, F., McBey, D., Harrison, M., Smith, P., Ehrhardt, F., Bhatia, A., Bellocchi, G., Brilli, L., Carozzi, M., Christie, K., Doltra, J., Dorich, C., Doro, L., Grace, P., Grant, B., Léonard, J., Liebig, M., Ludemann, C., Martin, R., Meier, E., Meyer, R., De Antoni Migliorati, M., Myrgiotis, V., Recous, S., Sándor, R., Snow, V., Soussana, J.-F., Smith, W.N., Fitton, N., How modelers model: the overlooked social and human dimensions in model Intercomparison studies. Environ. Sci. Technol. 56 (2022), 13485–13498, 10.1021/acs.est.2c02023.
Angulo, C., Rötter, R., Lock, R., Enders, A., Fronzek, S., Ewert, F., Implication of crop model calibration strategies for assessing regional impacts of climate change in Europe. Agric. For. Meteorol. 170 (2013), 32–46, 10.1016/j.agrformet.2012.11.017.
Angulo, C., Rötter, R., Trnka, M., Pirttioja, N., Gaiser, T., Hlavinka, P., Ewert, F., Characteristic ‘fingerprints’ of crop model responses to weather input data at different spatial resolutions. Eur. J. Agron. 49 (2013), 104–114, 10.1016/J.EJA.2013.04.003.
Asseng, S., Ewert, F., Rosenzweig, C., Jones, J.W., Hatfield, J.L., Ruane, A.C., Boote, K.J., Thorburn, P.J., Rötter, R.P., Cammarano, D., Brisson, N., Basso, B., Martre, P., Aggarwal, P.K., Angulo, C., Bertuzzi, P., Biernath, C., Challinor, A.J., Doltra, J., Gayler, S., Goldberg, R., Grant, R., Heng, L., Hooker, J., Hunt, L.A., Ingwersen, J., Izaurralde, R.C., Kersebaum, K.C., Müller, C., Naresh Kumar, S., Nendel, C., O'Leary, G., Olesen, J.E., Osborne, T.M., Palosuo, T., Priesack, E., Ripoche, D., Semenov, M.A., Shcherbak, I., Steduto, P., Stöckle, C., Stratonovitch, P., Streck, T., Supit, I., Tao, F., Travasso, M., Waha, K., Wallach, D., White, J.W., Williams, J.R., Wolf, J., Uncertainty in simulating wheat yields under climate change. Nat. Clim. Change 3 (2013), 827–832, 10.1038/nclimate1916.
Asseng, S., Martre, P., Maiorano, A., Rötter, R.P., O'Leary, G.J., Fitzgerald, G.J., Girousse, C., Motzo, R., Giunta, F., Babar, M.A., Reynolds, M.P., Kheir, A.M.S., Thorburn, P.J., Waha, K., Ruane, A.C., Aggarwal, P.K., Ahmed, M., Balkovič, J., Basso, B., Biernath, C., Bindi, M., Cammarano, D., Challinor, A.J., De Sanctis, G., Dumont, B., Eyshi Rezaei, E., Fereres, E., Ferrise, R., Garcia‐Vila, M., Gayler, S., Gao, Y., Horan, H., Hoogenboom, G., Izaurralde, R.C., Jabloun, M., Jones, C.D., Kassie, B.T., Kersebaum, K.-C., Klein, C., Koehler, A., Liu, B., Minoli, S., Montesino San Martin, M., Müller, C., Naresh Kumar, S., Nendel, C., Olesen, J.E., Palosuo, T., Porter, J.R., Priesack, E., Ripoche, D., Semenov, M.A., Stöckle, C., Stratonovitch, P., Streck, T., Supit, I., Tao, F., Van der Velde, M., Wallach, D., Wang, E., Webber, H., Wolf, J., Xiao, L., Zhang, Z., Zhao, Z., Zhu, Y., Ewert, F., Climate change impact and adaptation for wheat protein. Global Change Biol. 25 (2019), 155–173, 10.1111/gcb.14481.
Beaudoin, N., Lecharpentier, P., Ripoche-Wachter, D., Strullu, L., Mary, B.J.L., Launay, M., Justes, E., STICS Soil Crop Model. Conceptual Framework, Equations and Uses. 2023, Editions Quae, 10.35690/978-2-7592-3679-4.
Brewer, M.J., Butler, A., Cooksley, S.L., The relative performance of AIC, AICC and BIC in the presence of unobserved heterogeneity. Methods Ecol. Evol. 7 (2016), 679–692, 10.1111/2041-210X.12541.
Bruni, E., Chenu, C., Abramoff, R.Z., Baldoni, G., Barkusky, D., Clivot, H., Huang, Y., Kätterer, T., Pikuła, D., Spiegel, H., Virto, I., Guenet, B., Multi‐modelling predictions show high uncertainty of required carbon input changes to reach a 4‰ target. Eur. J. Soil Sci., 73, 2022, e13330, 10.1111/ejss.13330.
Buis, S., Lecharpentier, P., Vezy, R., Ginet, M., CroptimizR: a package to estimate parameters of crop models. [WWW Document] https://doi.org/10.5281/zenodo.4066451, 2023.
Ceglar, A., Črepinšek, Z., Kajfež-Bogataj, L., Pogačar, T., The simulation of phenological development in dynamic crop model: the Bayesian comparison of different methods. Agric. For. Meteorol. 151 (2011), 101–115, 10.1016/j.agrformet.2010.09.007.
Chakrabarti, A., Ghosh, J.K., AIC, BIC and recent advances in model selection. Philos. Stat, 2011, 583–605, 10.1016/B978-0-444-51862-0.50018-6.
Confalonieri, R., Orlando, F., Paleari, L., Stella, T., Gilardelli, C., Movedi, E., Pagani, V., Cappelli, G., Vertemara, A., Alberti, L., Alberti, P., Atanassiu, S., Bonaiti, M., Cappelletti, G., Ceruti, M., Confalonieri, A., Corgatelli, G., Corti, P., Dell'Oro, M., Ghidoni, A., Lamarta, A., Maghini, A., Mambretti, M., Manchia, A., Massoni, G., Mutti, P., Pariani, S., Pasini, D., Pesenti, A., Pizzamiglio, G., Ravasio, A., Rea, A., Santorsola, D., Serafini, G., Slavazza, M., Acutis, M., Uncertainty in crop model predictions: what is the role of users?. Environ. Model. Software 81 (2016), 165–173, 10.1016/j.envsoft.2016.04.009.
Doherty, J.E., Hunt, R.J., Tonkin, M.J., Approaches to Highly Parameterized Inversion: A Guide to Using PEST for Model-Parameter and Predictive-Uncertainty Analysis. U.S. Geological Survey Scientific Investigations Report 2010–5211, 2010.
Dumont, B., Leemans, V., Mansouri, M., Bodson, B., Destain, J.-P., Destain, M.-F., Parameter identification of the STICS crop model, using an accelerated formal MCMC approach. Environ. Model. Software 52 (2014), 121–135, 10.1016/j.envsoft.2013.10.022.
Guillaume, S., Berez, J.-E., Wallach, D., Justes, E., Methodological comparison of calibration procedures for durum wheat parameters in the STICS model. Eur. J. Agron. 35 (2011), 115–126.
Han, L., Neumann, M., Effect of dimensionality on the Nelder–Mead simplex method. Optim. Methods Softw. 21 (2006), 1–16, 10.1080/10556780512331318290.
He, D., Wang, E., Wang, J., Robertson, M.J., Data requirement for effective calibration of process-based crop models. Agric. For. Meteorol. 234–235 (2017), 136–148, 10.1016/j.agrformet.2016.12.015.
Iizumi, T., Yokozawa, M., Nishimori, M., Parameter estimation and uncertainty analysis of a large-scale crop model for paddy rice: application of a Bayesian approach. Agric. For. Meteorol. 149 (2009), 333–348.
Jansson, P.-E., CoupModel: model use, calibration, and validation. Trans. ASABE (Am. Soc. Agric. Biol. Eng.) 55 (2012), 1337–1346, 10.13031/2013.42245.
Jha, P.K., Ines, A.V.M., Han, E., Cruz, R., Vara Prasad, P.V., A comparison of multiple calibration and ensembling methods for estimating genetic coefficients of CERES-Rice to simulate phenology and yields. Field Crops Res., 284, 2022, 108560, 10.1016/J.FCR.2022.108560.
Jha, P.K., Ines, A.V.M., Singh, M.P., A multiple and ensembling approach for calibration and evaluation of genetic coefficients of CERES-Maize to simulate maize phenology and yield in Michigan. Environ. Model. Software, 135, 2021, 104901, 10.1016/j.envsoft.2020.104901.
Jones, J.W., He, J., Boote, K.J., Wilkens, P., Porter, C.H., Hu, Z., Ahuja, L.R., Ma, L., (eds.) Estimating DSSAT Cropping System Cultivar-specific Parameters Using Bayesian Techniques Methods of Introducing System Models into Agricultural Research, 2011, American Society of Agronomy, Madison, 365–394.
Keating, B., Carberry, P., Hammer, G., Probert, M., Robertson, M., Holzworth, D., Huth, N., Hargreaves, J.N., Meinke, H., Hochman, Z., McLean, G., Verburg, K., Snow, V., Dimes, J., Silburn, M., Wang, E., Brown, S., Bristow, K., Asseng, S., Chapman, S., McCown, R., Freebairn, D., Smith, C., An overview of APSIM, a model designed for farming systems simulation. Eur. J. Agron. 18 (2003), 267–288, 10.1016/S1161-0301(02)00108-9.
Kersebaum, K.C., Ahuja, L.R., Ma, L., (eds.) Special Features of the HERMES Model and Additional Procedures for Parameterization, Calibration, Validation, and Applications Methods of Introducing System Models into Agricultural Research, 2011, John Wiley & Sons, Ltd, 65–94, 10.2134/advagricsystmodel2.c2.
Lamboni, M., Makowski, D., Lehuger, S., Gabrielle, B., Monod, H., Multivariate global sensitivity analysis for dynamic crop models. Field Crops Res. 113 (2009), 312–320, 10.1016/j.fcr.2009.06.007.
Lecharpentier, P., Vezy, R., Buis, S., Ginet, M., STICSonR: Manage STICS Simulations Running the Executable or JavaStics. 2023, 10.5281/zenodo.8142442 [WWW Document].
Li, P., Ren, L., Evaluating the effects of limited irrigation on crop water productivity and reducing deep groundwater exploitation in the North China Plain using an agro-hydrological model: I. Parameter sensitivity analysis, calibration and model validation. J. Hydrol. 574 (2019), 497–516, 10.1016/J.JHYDROL.2019.04.053.
Li, T., Hasegawa, T., Yin, X., Zhu, Y., Boote, K., Adam, M., Bregaglio, S., Buis, S., Confalonieri, R., Fumoto, T., Gaydon, D., Marcaida, M., Nakagawa, H., Oriol, P., Ruane, A.C., Ruget, F., Singh, B.-, Singh, U., Tang, L., Tao, F., Wilkens, P., Yoshida, H., Zhang, Z., Bouman, B., Uncertainties in predicting rice yield by current crop models under a wide range of climatic conditions. Global Change Biol. 21 (2015), 1328–1341, 10.1111/gcb.12758.
López-Cruz, I.L., Ruiz-García, A., Fitz-Rodríguez, E., Salazar-Moreno, R., Rojano-Aguilar, A., A comparison of Bayesian and classical methods for parameter estimation in greenhouse crop models. Acta Hortic., 2017, 241–248, 10.17660/ActaHortic.2017.1182.29.
Maiorano, A., Martre, P., Asseng, S., Ewert, F., Müller, C., Rötter, R.P., Ruane, A.C., Semenov, M.A., Wallach, D., Wang, E., Alderman, P.D., Kassie, B.T., Biernath, C., Basso, B., Cammarano, D., Challinor, A.J., Doltra, J., Dumont, B., Rezaei, E.E., Gayler, S., Kersebaum, K.C., Kimball, B.A., Koehler, A.-K., Liu, B., O'Leary, G.J., Olesen, J.E., Ottman, M.J., Priesack, E., Reynolds, M., Stratonovitch, P., Streck, T., Thorburn, P.J., Waha, K., Wall, G.W., White, J.W., Zhao, Z., Zhu, Y., Crop model improvement reduces the uncertainty of the response to temperature of multi-model ensembles. Field Crops Res., 202, 2017, 10.1016/j.fcr.2016.05.001.
Martínez-Ruiz, A., Ruiz-García, A., Prado-Hernández, J.V., López-Cruz, I.L., Valencia-Islas, J.O., Pineda-Pineda, J., Global sensitivity analysis and calibration by differential evolution algorithm of HORTSYST crop model for fertigation management. Water, 13, 2021, 610, 10.3390/w13050610.
Martre, P., Wallach, D., Asseng, S., Ewert, F., Jones, J.W., Rötter, R.P., Boote, K.J., Ruane, A.C., Thorburn, P.J., Cammarano, D., Hatfield, J.L., Rosenzweig, C., Aggarwal, P.K., Angulo, C., Basso, B., Bertuzzi, P., Biernath, C., Brisson, N., Challinor, A.J., Doltra, J., Gayler, S., Goldberg, R., Grant, R.F., Heng, L., Hooker, J., Hunt, L.A., Ingwersen, J., Izaurralde, R.C., Kersebaum, K.C., Müller, C., Kumar, S.N., Nendel, C., O'leary, G., Olesen, J.E., Osborne, T.M., Palosuo, T., Priesack, E., Ripoche, D., Semenov, M.A., Shcherbak, I., Steduto, P., Stöckle, C.O., Stratonovitch, P., Streck, T., Supit, I., Tao, F., Travasso, M., Waha, K., White, J.W., Wolf, J., Multimodel ensembles of wheat growth: many models are better than one. Global Change Biol. 21 (2015), 911–925, 10.1111/gcb.12768.
Necpálová, M., Anex, R.P., Fienen, M.N., Del Grosso, S.J., Castellano, M.J., Sawyer, J.E., Iqbal, J., Pantoja, J.L., Barker, D.W., Understanding the DayCent model: calibration, sensitivity, and identifiability through inverse modeling. Environ. Model. Software 66 (2015), 110–130, 10.1016/J.ENVSOFT.2014.12.011.
Nelder, J.A., Mead, R., A simplex method for function minimization. Comput. J. 7 (1965), 308–313, 10.1093/comjnl/7.4.308.
Pasley, H., Brown, H., Holzworth, D., Whish, J., Bell, L., Huth, N., How to build a crop model. A review. Agron. Sustain. Dev., 43, 2023, 2, 10.1007/s13593-022-00854-9.
Pasquel, D., Roux, S., Richetti, J., Cammarano, D., Tisseyre, B., Taylor, J.A., A review of methods to evaluate crop model performance at multiple and changing spatial scales. Precis. Agric. 23 (2022), 1489–1513, 10.1007/s11119-022-09885-4.
Ramirez-Villegas, J., Koehler, A.-K., Challinor, A.J., Assessing uncertainty and complexity in regional-scale crop model simulations. Eur. J. Agron. 88 (2017), 84–95, 10.1016/J.EJA.2015.11.021.
Ramirez‐Villegas, J., Molero Milan, A., Alexandrov, N., Asseng, S., Challinor, A.J., Crossa, J., Eeuwijk, F., Ghanem, M.E., Grenier, C., Heinemann, A.B., Wang, J., Juliana, P., Kehel, Z., Kholova, J., Koo, J., Pequeno, D., Quiroz, R., Rebolledo, M.C., Sukumaran, S., Vadez, V., White, J.W., Reynolds, M., CGIAR modeling approaches for resource‐constrained scenarios: I. Accelerating crop breeding for a changing climate. Crop Sci. 60 (2020), 547–567, 10.1002/csc2.20048.
Rosenzweig, C., Jones, J.W., Hatfield, J.L., Ruane, A.C., Boote, K.J., Thorburn, P., Antle, J.M., Nelson, G.C., Porter, C., Janssen, S., Asseng, S., Basso, B., Ewert, F., Wallach, D., Baigorria, G., Winter, J.M., The agricultural model Intercomparison and improvement project (AgMIP): protocols and pilot studies. Agric. For. Meteorol., 170, 2013, 10.1016/j.agrformet.2012.09.011.
Seber, G.A.F., Wild, C.J., Nonlinear Regression. 1989, Wiley, New York.
Tao, F., Rötter, R.P., Palosuo, T., Gregorio Hernández Díaz‐Ambrona, C., Mínguez, M.I., Semenov, M.A., Kersebaum, K.C., Nendel, C., Specka, X., Hoffmann, H., Ewert, F., Dambreville, A., Martre, P., Rodríguez, L., Ruiz‐Ramos, M., Gaiser, T., Höhn, J.G., Salo, T., Ferrise, R., Bindi, M., Cammarano, D., Schulman, A.H., Contribution of crop model structure, parameters and climate projections to uncertainty in climate change impact assessments. Global Change Biol. 24 (2018), 1291–1307, 10.1111/gcb.14019.
van der Velde, M., Nisini, L., Performance of the MARS-crop yield forecasting system for the European Union: assessing accuracy, in-season, and year-to-year improvements from 1993 to 2015. Agric. Syst. 168 (2019), 203–212, 10.1016/J.AGSY.2018.06.009.
Vezy, R., Buis, S., Lecharpentier, P., Giner, M., CroPlotR: A Package to Analyse Crop Model Simulations Outputs with Plots and Statistics. 2023, 10.5281/zenodo.4066451 [WWW Document].
Wallach, D., Martre, P., Liu, B., Asseng, S., Ewert, F., Thorburn, P.J., Ittersum, M., Aggarwal, P.K., Ahmed, M., Basso, B., Biernath, C., Cammarano, D., Challinor, A.J., De Sanctis, G., Dumont, B., Eyshi Rezaei, E., Fereres, E., Fitzgerald, G.J., Gao, Y., Garcia-Vila, M., Gayler, S., Girousse, C., Hoogenboom, G., Horan, H., Izaurralde, R.C., Jones, C.D., Kassie, B.T., Kersebaum, C.C., Klein, C., Koehler, A.-K., Maiorano, A., Minoli, S., Müller, C., Naresh Kumar, S., Nendel, C., O'Leary, G.J., Palosuo, T., Priesack, E., Ripoche, D., Rötter, R.P., Semenov, M.A., Stöckle, C., Stratonovitch, P., Streck, T., Supit, I., Tao, F., Wolf, J., Zhang, Z., Multimodel ensembles improve predictions of crop–environment–management interactions. Global Change Biol., 2018, 10.1111/gcb.14411 0.
Wallach, D., Palosuo, T., Mielenz, H., Buis, S., Thorburn, P., Asseng, S., Dumont, B., Ferrise, R., Gayler, S., Ghahramani, A., Harrison, M.T., Hochman, Z., Hoogenboom, G., Huang, M., Jing, Q., Justes, E., Kersebaum, K.C., Launay, M., Lewan, E., Liu, K., Luo, Q., Mequanint, F., Nendel, C., Padovan, G., Olesen, J.E., Pullens, J.W.M., Qian, B., Seserman, D.-M., Shelia, V., Souissi, A., Specka, X., Wang, J., Weber, T.K.D., Weihermüller, L., Seidel, S.J., Uncertainty in Crop Phenology Simulations Is Driven Primarily by Parameter Variability. 2023, 10.1101/2023.02.03.526931 bioRxiv 2023.02.03.526931.
Wallach, D., Palosuo, T., Thorburn, P., Gourdain, E., Asseng, S., Basso, B., Buis, S., Crout, N., Dibari, C., Dumont, B., Ferrise, R., Gaiser, T., Garcia, C., Gayler, S., Ghahramani, A., Hochman, Z., Hoek, S., Hoogenboom, G., Horan, H., Huang, M., Jabloun, M., Jing, Q., Justes, E., Kersebaum, K.C., Klosterhalfen, A., Launay, M., Luo, Q., Maestrini, B., Mielenz, H., Moriondo, M., Nariman Zadeh, H., Olesen, J.E., Poyda, A., Priesack, E., Pullens, J.W.M., Qian, B., Schütze, N., Shelia, V., Souissi, A., Specka, X., Srivastava, A.K., Stella, T., Streck, T., Trombi, G., Wallor, E., Wang, J., Weber, T.K.D., Weihermüller, L., de Wit, A., Wöhling, T., Xiao, L., Zhao, C., Zhu, Y., Seidel, S.J., How well do crop modeling groups predict wheat phenology, given calibration data from the target population?. Eur. J. Agron., 124, 2021, 126195, 10.1016/j.eja.2020.126195.
Wallach, D., Palosuo, T., Thorburn, P., Hochman, Z., Andrianasolo, F., Asseng, S., Basso, B., Buis, S., Crout, N., Dumont, B., Ferrise, R., Gaiser, T., Gayler, S., Hiremath, S., Hoek, S., Horan, H., Hoogenboom, G., Huang, M., Jabloun, M., Jansson, P.-E., Jing, Q., Justes, E., Kersebaum, K.C., Launay, M., Lewan, E., Luo, Q., Maestrini, B., Moriondo, M., Olesen, J.E., Padovan, G., Poyda, A., Priesack, E., Pullens, J.W.M., Qian, B., Schütze, N., Shelia, V., Souissi, A., Specka, X., Kumar Srivastava, A., Stella, T., Streck, T., Trombi, G., Wallor, E., Wang, J., Weber, T.K.D., Weihermüller, L., de Wit, A., Wöhling, T., Xiao, L., Zhao, C., Zhu, Y., Seidel, S.J., Multi-model evaluation of phenology prediction for wheat in Australia. Agric. For. Meteorol., 298–299, 2021, 108289, 10.1016/J.AGRFORMET.2020.108289.
Wallach, D., Palosuo, T., Thorburn, P., Hochman, Z., Gourdain, E., Andrianasolo, F., Asseng, S., Basso, B., Buis, S., Crout, N., Dibari, C., Dumont, B., Ferrise, R., Gaiser, T., Garcia, C., Gayler, S., Ghahramani, A., Hiremath, S., Hoek, S., Horan, H., Hoogenboom, G., Huang, M., Jabloun, M., Jansson, P.-E., Jing, Q., Justes, E., Kersebaum, K.C., Klosterhalfen, A., Launay, M., Lewan, E., Luo, Q., Maestrini, B., Mielenz, H., Moriondo, M., Nariman Zadeh, H., Padovan, G., Olesen, J.E., Poyda, A., Priesack, E., Pullens, J.W.M., Qian, B., Schütze, N., Shelia, V., Souissi, A., Specka, X., Srivastava, A.K., Stella, T., Streck, T., Trombi, G., Wallor, E., Wang, J., Weber, T.K.D., Weihermüller, L., de Wit, A., Wöhling, T., Xiao, L., Zhao, C., Zhu, Y., Seidel, S.J., The chaos in calibrating crop models: lessons learned from a multi-model calibration exercise. Environ. Model. Software, 145, 2021, 105206, 10.1016/J.ENVSOFT.2021.105206.
Wallach, D., Palosuo, T., Thorburn, P., Mielenz, H., Buis, S., Hochman, Z., Gourdain, E., Andrianasolo, F., Dumont, B., Ferrise, R., Gaiser, T., Garcia, C., Gayler, S., Harrison, M., Hiremath, S., Horan, H., Hoogenboom, G., Jansson, P.-E., Jing, Q., Justes, E., Kersebaum, K.-C., Launay, M., Lewan, E., Liu, K., Mequanint, F., Moriondo, M., Nendel, C., Padovan, G., Qian, B., Schütze, N., Seserman, D.-M., Shelia, V., Souissi, A., Specka, X., Srivastava, A.K., Trombi, G., Weber, T.K.D., Weihermüller, L., Wöhling, T., Seidel, S.J., Proposal and extensive test of a calibration protocol for crop phenology models. Agron. Sustain. Dev., 43, 2023, 46, 10.1007/s13593-023-00900-0.
Wang, B., Feng, P., Liu, D.L., O'Leary, G.J., Macadam, I., Waters, C., Asseng, S., Cowie, A., Jiang, T., Xiao, D., Ruan, H., He, J., Yu, Q., Sources of uncertainty for wheat yield projections under future climate are site-specific. Nat. Food 1 (2020), 720–728, 10.1038/s43016-020-00181-w.
Wang, P.C., Shoup, T.E., Parameter sensitivity study of the Nelder–Mead simplex method. Adv. Eng. Software 42 (2011), 529–533, 10.1016/J.ADVENGSOFT.2011.04.004.
Wang, X., Kemanian, A., Williams, J., Ahuja, L.R., Ma, L., (eds.) Special Features of the EPIC and APEX Modeling Package and Procedures for Parameterization, Calibration, Validation, and Applications Methods of Introducing System Models into Agricultural Research, 2011, American Society of Agronomy, Madison, 177–208.
Webber, H., Asseng, S., Kimball, B., White, J., Ottman, M., Wall, G.W., De Sanctis, G., Doltra, J., Grant, R., Kassie, B., Maiorano, A., Olesen, J.E., Ripoche, D., Rezaei, E.E., Semenov, M.A., Stratonovitch, P., Canopy temperature for simulation of heat stress in irrigated wheat in a semi-arid environment: a multi-model comparison. Field Crops Res. 202 (2017), 21–35, 10.1016/J.FCR.2015.10.009.
Webber, H., Gaiser, T., Ewert, F., What role can crop models play in supporting climate change adaptation decisions to enhance food security in Sub-Saharan Africa?. Agric. Syst. 127 (2014), 161–177, 10.1016/J.AGSY.2013.12.006.
Xiong, W., Asseng, S., Hoogenboom, G., Hernandez-Ochoa, I., Robertson, R., Sonder, K., Pequeno, D., Reynolds, M., Gerard, B., Different uncertainty distribution between high and low latitudes in modelling warming impacts on wheat. Nat. Food 1 (2020), 63–69, 10.1038/s43016-019-0004-2.
Ypma, J., Johnson, S.G., Introduction to nloptr: an R interface to NLopt. [WWW Document]. URL https://astamm.github.io/nloptr/index.html, 2022.
Zhang, S., Tao, F., Zhang, Z., Uncertainty from model structure is larger than that from model parameters in simulating rice phenology in China. Eur. J. Agron. 87 (2017), 30–39, 10.1016/j.eja.2017.04.004.
