Crop model; Multi-model studies; Parameter uncertainty; Structure uncertainty; Forestry; Global and Planetary Change; Agronomy and Crop Science; Atmospheric Science
Abstract :
[en] It has become common to compare crop model results in multi-model simulation experiments. In general, one observes a large variability in such studies, which reduces the confidence one can have in such models. It is important to understand the causes of this variability as a first step toward reducing it. For a given data set, the variability in a multi-model study can arise from uncertainty in model structure or in parameter values for a given structure. Previous studies have made assumptions about the origin of parameter uncertainty, and then quantified its contribution, generally finding that parameter uncertainty is less important than structure uncertainty. However, those studies do not take account of the full parameter variability in multi-model studies. Here we propose estimating parameter uncertainty based on open-call multi-model ensembles where the same structure is used by more than one modeling group. The variability in such a case is due to the full variability of parameters among modeling groups. Then structure and parameter contributions can be estimated using random effects analysis of variance. Based on three multi-model studies for simulating wheat phenology, it is found that the contribution of parameter uncertainty to total uncertainty is, on average, more than twice as large as the uncertainty from structure. A second estimate, based on a comparison of two different calibration approaches for multiple models leads to a very similar result. We conclude that improvement of crop models requires as much attention to parameters as to model structure.
Disciplines :
Agriculture & agronomy Computer science
Author, co-author :
Wallach, Daniel ; Institute of Crop Science and Resource Conservation, University of Bonn, Germany
Palosuo, Taru ; Natural Resources Institute Finland (Luke), Helsinki, Finland
Mielenz, Henrike; Julius Kühn Institute (JKI) – Federal Research Centre for Cultivated Plants, Institute for Crop and Soil Science, Braunschweig, Germany
Buis, Samuel ; INRAE, Avignon, France
Thorburn, Peter ; CSIRO Agriculture and Food, Brisbane, Australia
Asseng, Senthold ; Technical University of Munich, Department of Life Science Engineering, Digital Agriculture, HEF World Agricultural Systems Center, Freising, Germany
Dumont, Benjamin ; Université de Liège - ULiège > TERRA Research Centre > Plant Sciences
Ferrise, Roberto; Department of Agriculture, Food, Environment and Forestry (DAGRI), University of Florence, Italy
Gayler, Sebastian ; Institute of Soil Science and Land Evaluation, Biogeophysics, University of Hohenheim, Stuttgart, Germany
Ghahramani, Afshin; Department of Environment and Science and Innovation, Queensland Government, Toowoomba, Australia
Harrison, Matthew Tom ; Tasmanian Institute of Agriculture, University of Tasmania, Launceston, Australia
Hochman, Zvi ; CSIRO Agriculture and Food, Brisbane, Australia
Hoogenboom, Gerrit ; Agricultural and Biological Engineering Department, University of Florida, Gainesville, Florida, USA & Global Food Systems Institute, University of Florida, Gainesville, United States
Huang, Mingxia; College of Resources and Environmental Sciences, China Agricultural University, China
Jing, Qi; Ottawa Research and Development Centre, Agriculture and Agri-Food Canada, Ottawa, Canada
Justes, Eric ; CIRAD, Univ Montpellier, Persyst Department, Montpellier, France
Kersebaum, Kurt Christian ; Leibniz Centre for Agricultural Landscape Research (ZALF), Müncheberg, Germany ; Global Change Research Institute CAS, Brno, Czech Republic ; Institute for Tropical Plant Production and Agricultural Systems Modelling, Georg-August University, Göttingen, Germany
Launay, Marie; INRAE, Avignon, France
Lewan, Elisabet ; Department of Soil and Environment, Swedish University of Agricultural Sciences (SLU), Uppsala, Sweden
Liu, Ke; Tasmanian Institute of Agriculture, University of Tasmania, Launceston, Australia
Luo, Qunying; Hillridge Technology Pty Ltd, Sydney, Australia
Mequanint, Fasil ; Institute of Soil Science and Land Evaluation, Biogeophysics, University of Hohenheim, Stuttgart, Germany
Nendel, Claas; Leibniz Centre for Agricultural Landscape Research (ZALF), Müncheberg, Germany ; Global Change Research Institute CAS, Brno, Czech Republic ; Institute of Biochemistry and Biology, University of Potsdam, Potsdam, Germany
Padovan, Gloria; Department of Agriculture, Food, Environment and Forestry (DAGRI), University of Florence, Italy
Olesen, Jørgen Eivind ; Department of Agroecology, Aarhus University, Tjele, Denmark
Pullens, Johannes Wilhelmus Maria ; Department of Agroecology, Aarhus University, Tjele, Denmark
Qian, Budong; Ottawa Research and Development Centre, Agriculture and Agri-Food Canada, Ottawa, Canada
Seserman, Diana-Maria; Leibniz Centre for Agricultural Landscape Research (ZALF), Müncheberg, Germany
Shelia, Vakhtang ; Agricultural and Biological Engineering Department, University of Florida, Gainesville, Florida, USA & Global Food Systems Institute, University of Florida, Gainesville, United States
Souissi, Amir; Swift Current Research and Development Centre, Agriculture and Agri-Food Canada, Swift Current, Canada
Specka, Xenia; Leibniz Centre for Agricultural Landscape Research (ZALF), Müncheberg, Germany
Wang, Jing; College of Resources and Environmental Sciences, China Agricultural University, China
Weber, Tobias K.D.; Soil Science Section, Faculty of Organic Agricultural Sciences, University of Kassel, Germany
Weihermüller, Lutz; Institute of Bio- and Geosciences - IBG-3, Agrosphere, Forschungszentrum Jülich GmbH, Jülich, Germany
Seidel, Sabine J.; Institute of Crop Science and Resource Conservation, University of Bonn, Germany ; Institute of Organic Farming, Department of Agricultural Sciences, University of Natural Resources and Life Sciences, Vienna, Austria
UniSQ - University of Southern Queensland NIFA - National Institute of Food and Agriculture MSMT - Ministerstvo školství, mládeže a tělovýchovy České republiky DFG - Deutsche Forschungsgemeinschaft Research Council of Finland Research Council of Finland USDA - United States Department of Agriculture BMBF - Bundesministerium für Bildung und Forschung CSC - China Scholarship Council INRA - Institut National de la Recherche Agronomique MiPAAF - Ministero delle Politiche Agricole, Alimentari e Forestali CSIRO - Commonwealth Scientific and Industrial Research Organisation
Funding text :
This work was in part supported by the Collaborative Research Center 1253 CAMPOS (Project 7: Stochastic Modelling Framework), funded by the German Research Foundation (DFG, Grant Agreement SFB 1253/1 2017 ), the Academy of Finland through projects AICropPro ( 316172 ) and DivCSA ( 316215 ), the BonaRes projects ''Soil3\u2032' ( BOMA 03037514 ) and \u201CI4S\u201D ( 031B053I ) of the Federal Ministry of Education and Research (BMBF) , Germany, the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany\u2019s Excellence Strategy - EXC 2070 \u2013 390732324 EXC (PhenoRob), the project BiomassWeb of the GlobeE programme (Grant number: FKZ031A258B ) funded by the Federal Ministry of Education and Research (BMBF, Germany), the INRA ACCAF meta-programme, the German Federal Ministry of Education and Research (BMBF) in the framework of the funding measure \u201CSoil as a Sustainable Resource for the Bioeconomy \u2013- BonaRes\u201D , project \u201CBonaRes (Module B): BonaRes Centre for Soil Research, subproject B\u201D (grant 031B0511B ), the National Key Research and Development Program of China ( 2017YFD0300205 ), the National Science Foundation for Distinguished Young Scholars ( 31725020 ), the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD) , the 111 project (B16026), and China Scholarship Council, the Agriculture and Agri-Food Canada's Project J-002303 under the Interdepartmental Research Initiative in Agriculture, the DFG Research Unit FOR 1695 \u2018Agricultural Landscapes under Global Climate Change \u2013 Processes and Feedbacks on a Regional Scale, the U.S. Department of Agriculture National Institute of Food and Agriculture (award no. 2015-68007-23133 ) and USDA/NIFA HATCH grant N. MCL02368, the National Key Research and Development Program of China ( 2016YFD0300105 ), The Broadacre Agriculture Initiative, a research partnership between University of Southern Queensland and the Queensland Department of Agriculture and Fisheries, the Academy of Finland through project AI-CropPro (315896), the JPI FACCE MACSUR2 project, funded by the Italian Ministry for Agricultural, Food, and Forestry Policies (D.M. 24064/7303/15 of 6/Nov/2015), the SYSTEMIC funded by JPI HDHL, JPI-OCEANS and FACCE-JPI under ERA-NET (n.696295), the SustEs project funded by the Ministry of Education, Youth and Sports of the Czech Republic (CZ.02.1.01/0.0/0.0/16_019/000797). The time of MTH and KL was supported by the Grains Research and Development Corporation (Project UOT1906-002RTX). The observed Australian phenology data were jointly funded by CSIRO, the Grains Research and Development Corporation (GRDC) under the \"Adding Value to GRDC\u2019s National Variety Trial Network\" project (CSA00027). The order in which the donors are listed is arbitrary.
Ahmad, S., Abbas, G., Ahmed, M., Fatima, Z., Anjum, M.A., Rasul, G., Khan, M.A., Hoogenboom, G., Climate warming and management impact on the change of phenology of the rice-wheat cropping system in Punjab. Pak. F. Crop. Res. 230 (2019), 46–61, 10.1016/j.fcr.2018.10.008.
Ahuja, L.R., Ma, L., A synthesis of current parameterization approaches and needs for further improvements. Methods of Introducing System Models into Agricultural Research, Advances in Agricultural Systems Modeling, 2011, 427–440, 10.2134/advagricsystmodel2.c15.
Albanito, F., McBey, D., Harrison, M., Smith, P., Ehrhardt, F., Bhatia, A., Bellocchi, G., Brilli, L., Carozzi, M., Christie, K., Doltra, J., Dorich, C., Doro, L., Grace, P., Grant, B., Léonard, J., Liebig, M., Ludemann, C., Martin, R., Meier, E., Meyer, R., De Antoni Migliorati, M., Myrgiotis, V., Recous, S., Sándor, R., Snow, V., Soussana, J.-F., Smith, W.N., Fitton, N., How modelers model: the overlooked social and Human dimensions in model intercomparison studies. Env. Sci. Technol. 56 (2022), 13485–13498, 10.1021/acs.est.2c02023.
Asseng, S., Ewert, F., Rosenzweig, C., Jones, J.W., Hatfield, J.L., Ruane, A.C., Boote, K.J., Thorburn, P.J., Rötter, R.P., Cammarano, D., Brisson, N., Basso, B., Martre, P., Aggarwal, P.K., Angulo, C., Bertuzzi, P., Biernath, C., Challinor, A.J., Doltra, J., Gayler, S., Goldberg, R., Grant, R., Heng, L., Hooker, J., Hunt, L.A., Ingwersen, J., Izaurralde, R.C., Kersebaum, K.C., Müller, C., Naresh Kumar, S., Nendel, C., O'Leary, G., Olesen, J.E., Osborne, T.M., Palosuo, T., Priesack, E., Ripoche, D., Semenov, M.A., Shcherbak, I., Steduto, P., Stöckle, C., Stratonovitch, P., Streck, T., Supit, I., Tao, F., Travasso, M., Waha, K., Wallach, D., White, J.W., Williams, J.R., Wolf, J., Uncertainty in simulating wheat yields under climate change. Nat. Clim. Chang. 3 (2013), 827–832, 10.1038/nclimate1916.
Bates, D., Mächler, M., Bolker, B., Walker, S., Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67 (2015), 1–48, 10.18637/jss.v067.i01.
Bindi, M., Palosuo, T., Trnka, M., Semenov, M., Modelling climate change impacts on crop production for food security. Clim. Res. 65 (2015), 3–5, 10.3354/cr01342.
Boote, K.J., Jones, J.W., Hoogenboom, G., White, J.W., The role of crop systems simulation in agriculture and environment. Int. J. Agric. Env. Inf. Syst. 1 (2010), 41–54, 10.4018/978-1-4666-0333-2.ch018.
Butts, M.B., Payne, J.T., Kristensen, M., Madsen, H., An evaluation of the impact of model structure on hydrological modelling uncertainty for streamflow simulation. J. Hydrol. 298 (2004), 242–266, 10.1016/J.JHYDROL.2004.03.042.
Confalonieri, R., Orlando, F., Paleari, L., Stella, T., Gilardelli, C., Movedi, E., Pagani, V., Cappelli, G., Vertemara, A., Alberti, L., Alberti, P., Atanassiu, S., Bonaiti, M., Cappelletti, G., Ceruti, M., Confalonieri, A., Corgatelli, G., Corti, P., Dell'Oro, M., Ghidoni, A., Lamarta, A., Maghini, A., Mambretti, M., Manchia, A., Massoni, G., Mutti, P., Pariani, S., Pasini, D., Pesenti, A., Pizzamiglio, G., Ravasio, A., Rea, A., Santorsola, D., Serafini, G., Slavazza, M., Acutis, M., Uncertainty in crop model predictions: what is the role of users?. Env. Model. Softw. 81 (2016), 165–173, 10.1016/j.envsoft.2016.04.009.
Durand, J.-L., Delusca, K., Boote, K., Lizaso, J., Manderscheid, R., Weigel, H.J., Ruane, A.C., Rosenzweig, C., Jones, J., Ahuja, L., Anapalli, S., Basso, B., Baron, C., Bertuzzi, P., Biernath, C., Deryng, D., Ewert, F., Gaiser, T., Gayler, S., Heinlein, F., Kersebaum, K.C., Kim, S.-H., Müller, C., Nendel, C., Olioso, A., Priesack, E., Villegas, J.R., Ripoche, D., Rötter, R.P., Seidel, S.I., Srivastava, A., Tao, F., Timlin, D., Twine, T., Wang, E., Webber, H., Zhao, Z., How accurately do maize crop models simulate the interactions of atmospheric CO2 concentration levels with limited water supply on water use and yield?. Eur. J. Agron. 100 (2018), 67–75, 10.1016/J.EJA.2017.01.002.
Fatima, Z., Ahmed, M., Hussain, M., Abbas, G., Ul-Allah, S., Ahmad, S., Ahmed, N., Ali, M.A., Sarwar, G., Haque, E.ul, Iqbal, P., Hussain, S., The fingerprints of climate warming on cereal crops phenology and adaptation options. Sci. Rep., 10, 2020, 18013, 10.1038/s41598-020-74740-3.
Gao, Y., Wallach, D., Liu, B., Dingkuhn, M., Boote, K.J., Singh, U., Asseng, S., Kahveci, T., He, J., Zhang, R., Confalonieri, R., Hoogenboom, G., Comparison of three calibration methods for modeling rice phenology. Agric. For. Meteorol., 280, 2020, 107785, 10.1016/J.AGRFORMET.2019.107785.
Harrison, M.T., Roggero, P.P., Zavattaro, L., Simple, efficient and robust techniques for automatic multi-objective function parameterisation: case studies of local and global optimisation using APSIM. Env. Model. Softw. 117 (2019), 109–133, 10.1016/j.envsoft.2019.03.010.
Houska, T., Kraft, P., Chamorro-Chavez, A., Breuer, L., SPOTting model parameters using a ready-made Python package. PLoS. One, 10, 2015, e0145180, 10.1371/journal.pone.0145180.
Iizumi, T., Yokozawa, M., Nishimori, M., Parameter estimation and uncertainty analysis of a large-scale crop model for paddy rice: application of a bayesian approach. Agric. For. Meteorol. 149 (2009), 333–348, 10.1016/j.agrformet.2008.08.015.
Liu, Y., Bachofen, C., Wittwer, R., Silva Duarte, G., Sun, Q., Klaus, V.H., Buchmann, N., Using PhenoCams to track crop phenology and explain the effects of different cropping systems on yield. Agric. Syst., 195, 2022, 103306, 10.1016/J.AGSY.2021.103306.
Maiorano, A., Martre, P., Asseng, S., Ewert, F., Müller, C., Rötter, R.P., Ruane, A.C., Semenov, M.A., Wallach, D., Wang, E., Alderman, P.D., Kassie, B.T., Biernath, C., Basso, B., Cammarano, D., Challinor, A.J., Doltra, J., Dumont, B., Rezaei, E.E., Gayler, S., Kersebaum, K.C., Kimball, B.A., Koehler, A.-K., Liu, B., O'Leary, G.J., Olesen, J.E., Ottman, M.J., Priesack, E., Reynolds, M., Stratonovitch, P., Streck, T., Thorburn, P.J., Waha, K., Wall, G.W., White, J.W., Zhao, Z., Zhu, Y., Crop model improvement reduces the uncertainty of the response to temperature of multi-model ensembles. F. Crop. Res., 202, 2017, 10.1016/j.fcr.2016.05.001.
Meier, U., 1997. Growth stages of mono- and dicotyledonous plants: BBCH-Monograph [WWW Document]. URL https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=2ahUKEwjBo-Ln4_r4AhVM3RoKHc-TA5sQFnoECAcQAQ&url=https%3A%2F%2Fwww.politicheagricole.it%2Fflex%2FAppData%2FWebLive%2FAgrometeo%2FMIEPFY800%2FBBCHengl2001.pdf&usg=AOvVaw1G_eUh_-qnVt91_qbj44d (accessed 1.9.19).
Migliavacca, M., Sonnentag, O., Keenan, T.F., Cescatti, A., O'Keefe, J., Richardson, A.D., On the uncertainty of phenological responses to climate change, and implications for a terrestrial biosphere model. Biogeosciences 9 (2012), 2063–2083, 10.5194/bg-9-2063-2012.
Muleke, A., Harrison, M.T., de Voil, P., Hunt, I., Liu, K., Yanotti, M., Eisner, R., Earlier crop flowering caused by global warming alleviated by irrigation. Env. Res. Lett., 17, 2022, 044032, 10.1088/1748-9326/ac5a66.
Müller, C., Franke, J., Jägermeyr, J., Ruane, A.C., Elliott, J., Moyer, E., Heinke, J., Falloon, P.D., Folberth, C., Francois, L., Hank, T., Izaurralde, R.C., Jacquemin, I., Liu, W., Olin, S., Pugh, T.A.M., Williams, K., Zabel, F., Exploring uncertainties in global crop yield projections in a large ensemble of crop models and CMIP5 and CMIP6 climate scenarios. Env. Res. Lett., 16, 2021, 034040, 10.1088/1748-9326/abd8fc.
R Core Team, 2017. A language and environment for statistical computing.
Ramirez-Villegas, J., Koehler, A.-K., Challinor, A.J., Assessing uncertainty and complexity in regional-scale crop model simulations. Eur. J. Agron. 88 (2017), 84–95, 10.1016/J.EJA.2015.11.021.
Ricciuto, D.M., Yang, X., Wang, D., Thornton, P.E., 2021. The impacts of model structure, parameter uncertainty and experimental design on earth system model simulations of litter bag decomposition experiments. https://doi.org/10.5194/bg-2021-163.
Rosenzweig, C., Jones, J.W., Hatfield, J.L., Ruane, A.C., Boote, K.J., Thorburn, P., Antle, J.M., Nelson, G.C., Porter, C., Janssen, S., Asseng, S., Basso, B., Ewert, F., Wallach, D., Baigorria, G., Winter, J.M., The Agricultural Model Intercomparison and Improvement Project (AgMIP): protocols and pilot studies. Agric. For. Meteorol., 170, 2013, 10.1016/j.agrformet.2012.09.011.
Rötter, R.P., Carter, T.R., Olesen, J.E., Porter, J.R., Crop–climate models need an overhaul. Nat. Clim. Chang. 1 (2011), 175–177, 10.1038/nclimate1152.
Salo, T.J., Palosuo, T., Kersebaum, K.C., Nendel, C., Angulo, C., Ewert, F., Bindi, M., Calanca, P., Klein, T., Moriondo, M., Ferrise, R., Olesen, J.E., Patil, R.H., Ruget, F., Takac, J., Hlavinka, P., Trnka, M., Rotter, R.P., Comparing the performance of 11 crop simulation models in predicting yield response to nitrogen fertilization. J. Agric. Sci. 154 (2016), 1218–1240, 10.1017/S0021859615001124.
Sándor, R., Ehrhardt, F., Grace, P., Recous, S., Smith, P., Snow, V., Soussana, J.-F., Basso, B., Bhatia, A., Brilli, L., Doltra, J., Dorich, C.D., Doro, L., Fitton, N., Grant, B., Harrison, M.T., Kirschbaum, M.U.F., Klumpp, K., Laville, P., Léonard, J., Martin, R., Massad, R.-S., Moore, A., Myrgiotis, V., Pattey, E., Rolinski, S., Sharp, J., Skiba, U., Smith, W., Wu, L., Zhang, Q., Bellocchi, G., Ensemble modelling of carbon fluxes in grasslands and croplands. F. Crop. Res., 252, 2020, 107791, 10.1016/j.fcr.2020.107791.
Scheffé, H., The analysis of variance. 1959, John Wiley & Sons, New York.
Seber, G.A.F., Wild, C.J., Nonlinear regression. 1989, Wiley, New York.
Seidel, S.J., Palosuo, T., Thorburn, P., Wallach, D., Towards improved calibration of crop models – Where are we now and where should we go?. Eur. J. Agron. 94 (2018), 25–35, 10.1016/J.EJA.2018.01.006.
Smallman, T.L., Milodowski, D.T., Neto, E.S., Koren, G., Ometto, J., Williams, M., Parameter uncertainty dominates C-cycle forecast errors over most of Brazil for the 21st century. Earth Syst. Dynam. 12 (2021), 1191–1237, 10.5194/esd-12-1191-2021.
Tao, F., Rötter, R.P., Palosuo, T., Gregorio Hernández Díaz-Ambrona, C., Mínguez, M.I., Semenov, M.A., Kersebaum, K.C., Nendel, C., Specka, X., Hoffmann, H., Ewert, F., Dambreville, A., Martre, P., Rodríguez, L., Ruiz-Ramos, M., Gaiser, T., Höhn, J.G., Salo, T., Ferrise, R., Bindi, M., Cammarano, D., Schulman, A.H., Contribution of crop model structure, parameters and climate projections to uncertainty in climate change impact assessments. Glob. Chang. Biol. 24 (2018), 1291–1307, 10.1111/gcb.14019.
Tebaldi, C., Knutti, R., The use of the multi-model ensemble in probabilistic climate projections. Philos. Trans. a Math. Phys. Eng. Sci. 365 (2007), 2053–2075, 10.1098/rsta.2007.2076.
Thébault, C., Perrin, C., Andréassian, V., Thirel, G., Legrand, S., Delaigue, O., Multi-model approach in a variable spatial framework for streamflow simulation. Hydrol. Earth. Syst. Sci. 28 (2024), 1539–1566, 10.5194/hess-28-1539-2024.
Wallach, D., Buis, S., Seserman, D.-M., Palosuo, T., Thorburn, P.J., Mielenz, H., Justes, E., Kersebaum, K.-C., Dumont, B., Launay, M., Seidel, S.J., A calibration protocol for soil-crop models. Env. Model. Softw., 180, 2024, 106147, 10.1016/J.ENVSOFT.2024.106147.
Wallach, D., Kim, K.S., Hyun, S., Buis, S., Thorburn, P., Mielenz, H., Seidel, S.J., Alderman, P.D., Dumont, B., Fallah, M.H., Hoogenboom, G., Justes, E., Kersebaum, K.-C., Launay, M., Leolini, L., Mehmood, M.Z., Moriondo, M., Jing, Q., Qian, B., Susanne, S., Seserman, D.-M., Shelia, V., Weihermüller, L., Palosuo, T., Evaluating the AgMIP calibration protocol for crop models; case study and new diagnostic tests. Eur. J. Agron., 168, 2025, 127659, 10.1016/J.EJA.2025.127659.
Wallach, D., Nissanka, S.P., Karunaratne, A.S., Weerakoon, W.M.W., Thorburn, P.J., Boote, K.J., Jones, J.W., Accounting for both parameter and model structure uncertainty in crop model predictions of phenology: A case study on rice. Eur. J. Agron., 88, 2017, 10.1016/j.eja.2016.05.013.
Wallach, D., Palosuo, T., Thorburn, P., Gourdain, E., Asseng, S., Basso, B., Buis, S., Crout, N., Dibari, C., Dumont, B., Ferrise, R., Gaiser, T., Garcia, C., Gayler, S., Ghahramani, A., Hochman, Z., Hoek, S., Hoogenboom, G., Horan, H., Huang, M., Jabloun, M., Jing, Q., Justes, E., Kersebaum, K.C., Klosterhalfen, A., Launay, M., Luo, Q., Maestrini, B., Mielenz, H., Moriondo, M., Nariman Zadeh, H., Olesen, J.E., Poyda, A., Priesack, E., Pullens, J.W.M., Qian, B., Schütze, N., Shelia, V., Souissi, A., Specka, X., Srivastava, A.K., Stella, T., Streck, T., Trombi, G., Wallor, E., Wang, J., Weber, T.K.D., Weihermüller, L., de Wit, A., Wöhling, T., Xiao, L., Zhao, C., Zhu, Y., Seidel, S.J., How well do crop modeling groups predict wheat phenology, given calibration data from the target population?. Eur. J. Agron., 124, 2021, 126195, 10.1016/j.eja.2020.126195.
Wallach, D., Palosuo, T., Thorburn, P., Hochman, Z., Andrianasolo, F., Asseng, S., Basso, B., Buis, S., Crout, N., Dumont, B., Ferrise, R., Gaiser, T., Gayler, S., Hiremath, S., Hoek, S., Horan, H., Hoogenboom, G., Huang, M., Jabloun, M., Jansson, P.-E., Jing, Q., Justes, E., Kersebaum, K.C., Launay, M., Lewan, E., Luo, Q., Maestrini, B., Moriondo, M., Olesen, J.E., Padovan, G., Poyda, A., Priesack, E., Pullens, J.W.M., Qian, B., Schütze, N., Shelia, V., Souissi, A., Specka, X., Kumar Srivastava, A., Stella, T., Streck, T., Trombi, G., Wallor, E., Wang, J., Weber, T.K.D., Weihermüller, L., de Wit, A., Wöhling, T., Xiao, L., Zhao, C., Zhu, Y., Seidel, S.J., Multi-model evaluation of phenology prediction for wheat in Australia. Agric. For. Meteorol., 298–299, 2021, 108289, 10.1016/J.AGRFORMET.2020.108289.
Wallach, D., Palosuo, T., Thorburn, P., Hochman, Z., Gourdain, E., Andrianasolo, F., Asseng, S., Basso, B., Buis, S., Crout, N., Dibari, C., Dumont, B., Ferrise, R., Gaiser, T., Garcia, C., Gayler, S., Ghahramani, A., Hiremath, S., Hoek, S., Horan, H., Hoogenboom, G., Huang, M., Jabloun, M., Jansson, P.-E., Jing, Q., Justes, E., Kersebaum, K.C., Klosterhalfen, A., Launay, M., Lewan, E., Luo, Q., Maestrini, B., Mielenz, H., Moriondo, M., Nariman Zadeh, H., Padovan, G., Olesen, J.E., Poyda, A., Priesack, E., Pullens, J.W.M., Qian, B., Schütze, N., Shelia, V., Souissi, A., Specka, X., Srivastava, A.K., Stella, T., Streck, T., Trombi, G., Wallor, E., Wang, J., Weber, T.K.D., Weihermüller, L., de Wit, A., Wöhling, T., Xiao, L., Zhao, C., Zhu, Y., Seidel, S.J., The chaos in calibrating crop models: lessons learned from a multi-model calibration exercise. Env. Model. Softw, 145, 2021, 105206, 10.1016/J.ENVSOFT.2021.105206.
Wallach, D., Palosuo, T., Thorburn, P., Mielenz, H., Buis, S., Hochman, Z., Gourdain, E., Andrianasolo, F., Dumont, B., Ferrise, R., Gaiser, T., Garcia, C., Gayler, S., Harrison, M., Hiremath, S., Horan, H., Hoogenboom, G., Jansson, P.-E., Jing, Q., Justes, E., Kersebaum, K.-C., Launay, M., Lewan, E., Liu, K., Mequanint, F., Moriondo, M., Nendel, C., Padovan, G., Qian, B., Schütze, N., Seserman, D.-M., Shelia, V., Souissi, A., Specka, X., Srivastava, A.K., Trombi, G., Weber, T.K.D., Weihermüller, L., Wöhling, T., Seidel, S.J., Proposal and extensive test of a calibration protocol for crop phenology models. Agron. Sustain. Dev., 43, 2023, 46, 10.1007/s13593-023-00900-0.
Wang, B., Feng, P., Liu, D.L., O'Leary, G.J., Macadam, I., Waters, C., Asseng, S., Cowie, A., Jiang, T., Xiao, D., Ruan, H., He, J., Yu, Q., Sources of uncertainty for wheat yield projections under future climate are site-specific. Nat. Food 1 (2020), 720–728, 10.1038/s43016-020-00181-w.
Wang, B., Jägermeyr, J., O'Leary, G.J., Wallach, D., Ruane, A.C., Feng, P., Li, L., Liu, D.L., Waters, C., Yu, Q., Asseng, S., Rosenzweig, C., Pathways to identify and reduce uncertainties in agricultural climate impact assessments. Nat. Food, 2024, 1–7, 10.1038/s43016-024-01014-w.
Xiong, W., Asseng, S., Hoogenboom, G., Hernandez-Ochoa, I., Robertson, R., Sonder, K., Pequeno, D., Reynolds, M., Gerard, B., Different uncertainty distribution between high and low latitudes in modelling warming impacts on wheat. Nat. Food 1 (2020), 63–69, 10.1038/s43016-019-0004-2.
Zadeh, F.K., Nossent, J., Bauwens, W., Van Griensven, A., A fast and effective parameterization of water quality models. Env. Model. Softw., 149, 2022, 105331, 10.1016/j.envsoft.2022.105331.
Zhang, S., Tao, F., Shi, R., Modeling the rice phenology and production in China with SIMRIW: sensitivity analysis and parameter estimation. Front. Earth. Sci. 8 (2014), 505–511, 10.1007/s11707-014-0468-1.
Zhang, S., Tao, F., Zhang, Z., Uncertainty from model structure is larger than that from model parameters in simulating rice phenology in China. Eur. J. Agron. 87 (2017), 30–39, 10.1016/j.eja.2017.04.004.
