The chaos in calibrating crop models: Lessons learned from a multi-model calibration exercise

[en] Calibration, the estimation of model parameters based on fitting the model to experimental data, is among the first steps in many applications of process-based models and has an important impact on simulated values. We propose a novel method of developing guidelines for calibration of process-based models, based on development of recommendations for calibration of the phenology component of crop models. The approach was based on a multi-model study, where all teams were provided with the same data and asked to return simulations for the same conditions. All teams were asked to document in detail their calibration approach, including choices with respect to criteria for best parameters, choice of parameters to estimate and software. Based on an analysis of the advantages and disadvantages of the various choices, we propose calibration recommendations that cover a comprehensive list of decisions and that are based on actual practices.

Disciplines :

Agriculture & agronomy
Computer science

Author, co-author :

Wallach, Daniel

Palosuo, Taru

Thorburn, Peter

Hochman, Zvi

Gourdain, Emmanuelle

Andrianasolo, Fety

Asseng, Senthold

Basso, Bruno

Buis, Samuel

Crout, Neil

More authors (49 more)

Language :

English

Title :

The chaos in calibrating crop models: Lessons learned from a multi-model calibration exercise

Publication date :

2021

Journal title :

Environmental Modelling & Software

ISSN :

1364-8152

Volume :

145

Pages :

105206

Peer reviewed :

Peer reviewed

Additional URL :

https://www.sciencedirect.com/science/article/pii/S1364815221002486

Available on ORBi :

since 01 December 2021

Statistics

Number of views

68 (3 by ULiège)

Number of downloads

73 (2 by ULiège)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

OpenAlex citations

Bibliography

Ahuja, L.R., Ma, L., Ahuja, Laj R., Ma, L., A synthesis of current parameterization approaches and needs for further improvements. Methods of Introducing System Models into Agricultural Research, 2011, American Society of Agronomy, Crop Science Society of America, Soil Science Society of America, 427–440, 10.2134/advagricsystmodel2.c15.
Akaike, H., Information theory and an extension of the maximum likelihood principle. Petrov, B.N., Csaki, F., (eds.) Proceedings of the 2nd International Symposium on Information Theory, 1973, Akademiai Kiado, Budapest, 267–281 B. N. Petrov, & F. Csaki.
Angulo, C., Rötter, R., Lock, R., Enders, A., Fronzek, S., Implication of crop model calibration strategies for assessing regional impacts of climate change in Europe. Agric. For., 2013.
Anothai, J., Patanothai, A., Jogloy, S., Pannangpetch, K., Boote, K.J., Hoogenboom, G., A sequential approach for determining the cultivar coefficients of peanut lines using end-of-season data of crop performance trials. Field Crop. Res. 108 (2008), 169–178.
Badham, J., Elsawah, S., Guillaume, J.H.A., Hamilton, S.H., Hunt, R.J., Jakeman, A.J., Pierce, S.A., Snow, V.O., Babbar-Sebens, M., Fu, B., Gober, P., Hill, M.C., Iwanaga, T., Loucks, D.P., Merritt, W.S., Peckham, S.D., Richmond, A.K., Zare, F., Ames, D., Bammer, G., Effective modeling for Integrated Water Resource Management: a guide to contextual practices by phases and steps and future opportunities. Environ. Model. Software 116 (2019), 40–56, 10.1016/J.ENVSOFT.2019.02.013.
Berger, J.O., Bayarri, M.J., The interplay of Bayesian and frequentist analysis. Stat. Sci. 19 (2004), 58–80, 10.1214/088342304000000116.
Beven, K., Freer, J., Equifinality, data assimilation, and uncertainty estimation in mechanistic modelling of complex environmental systems using the GLUE methodology. J. Hydrol. 249 (2001), 11–29, 10.1016/S0022-1694(01)00421-8.
Bhar, A., Kumar, R., Qi, Z., Malone, R., Coordinate descent based agricultural model calibration and optimized input management. Comput. Electron. Agric., 172, 2020, 105353, 10.1016/J.COMPAG.2020.105353.
Blasone, R.S., Madsen, H., Rosbjerg, D., Calibration and reliability in groundwater modelling. Bierkens, M.F., Gehrels, J.C., Kovarik, K., (eds.) Calibration and Reliability in Groundwater Modelling: from Uncertainty to Decision Making (Proceedings of ModelCARE'2005, the Hague, The Netherlands, June 2005), 2006, IAHS, 316.
Buddhaboon, C., Jintrawet, A., Hoogenboom, G., Methodology to estimate rice genetic coefficients for the CSM-CERES-Rice model using GENCALC and GLUE genetic coefficient estimators. J. Agric. Sci. 156 (2018), 482–492, 10.1017/S0021859618000527.
Buis, S., Wallach, D., Guillaume, S., Varella, H., Lecharpentier, P., Launay, M., Guérif, M., Bergez, J.-E., Justes, E., The STICS Crop Model and Associated Software for Analysis, Parameterization, and Evaluation. 2011, John Wiley & Sons, Ltd, 395–426.
Burnham, K.P., Anderson, D.R., Huyvaert, K.P., AIC model selection and multimodel inference in behavioral ecology: some background, observations, and comparisons. Behav. Ecol. Sociobiol. 65 (2011), 23–35, 10.1007/s00265-010-1029-6.
Doherty, J., Hunt, R.J., Two statistics for evaluating parameter identifiability and error reduction. J. Hydrol. 366 (2009), 119–127, 10.1016/J.JHYDROL.2008.12.018.
Doherty, J.E., Hunt, R.J., Tonkin, M.J., Approaches to Highly Parameterized Inversion: A Guide to Using PEST for Model-Parameter and Predictive-Uncertainty Analysis. 2010, U.S. Geological Survey Scientific Investigations Report 2010–5211.
Donatelli, M., Magarey, R.D., Bregaglio, S., Willocquet, L., Whish, J.P.M., Savary, S., Modelling the impacts of pests and diseases on agricultural systems. Agric. Syst. 155 (2017), 213–224, 10.1016/J.AGSY.2017.01.019.
Efstratiadis, A., Koutsoyiannis, D., One decade of multi-objective calibration approaches in hydrological modelling: a review. Hydrol. Sci. J. 55 (2010), 58–78, 10.1080/02626660903526292.
Ewert, F., Rötter, R.P., Bindi, M., Webber, H., Trnka, M., Kersebaum, K.C., Olesen, J.E., van Ittersum, M.K., Janssen, S., Rivington, M., Semenov, M.A., Wallach, D., Porter, J.R., Stewart, D., Verhagen, J., Gaiser, T., Palosuo, T., Tao, F., Nendel, C., Roggero, P.P., Bartošová, L., Asseng, S., Crop modelling for integrated assessment of risk to food production from climate change. Environ. Model. Software, 72, 2015, 10.1016/j.envsoft.2014.12.003.
Fath, B., Jorgensen, S.E., Fundamentals of Ecological Modelling: Applications in Environmental Management and Research. fourth ed., 2011, Elsevier, Amsterdam.
Franchini, M., Galeati, G., Comparing several genetic algorithm schemes for the calibration of conceptual rainfall-runoff models. Hydrol. Sci. J. 42 (1997), 357–379.
Gao, Y., Wallach, D., Liu, B., Dingkuhn, M., Boote, K.J., Singh, U., Asseng, S., Kahveci, T., He, J., Zhang, R., Confalonieri, R., Hoogenboom, G., Comparison of three calibration methods for modeling rice phenology. Agric. For. Meteorol., 280, 2020, 107785, 10.1016/J.AGRFORMET.2019.107785.
Guillaume, S., Berez, J.-E., Wallach, D., Justes, E., Methodological comparison of calibration procedures for durum wheat parameters in the STICS model. Eur. J. Agron. 35 (2011), 115–126.
He, J., Jones, J.W., Graham, W.D., Dukes, M.D., Influence of likelihood function choice for estimating crop model parameters using the generalized likelihood uncertainty estimation method. Agric. Syst. 103 (2010), 256–264, 10.1016/j.agsy.2010.01.006.
Hunt, J.R., Lilley, J.M., Trevaskis, B., Flohr, B.M., Peake, A., Fletcher, A., Zwart, A.B., Gobbett, D., Kirkegaard, J.A., Early sowing systems can boost Australian wheat yields despite recent climate change. Nat. Clim. Change 9 (2019), 244–247, 10.1038/s41558-019-0417-9.
Hunt, L.A., Pararajasingham, S., Jones, J.W., Hoogenboom, G., Imamura, D.T., Ogoshi, R.M., GENCALC: software to facilitate the use of crop models for analyzing field experiments. Agron. J. 85 (1993), 1090–1094.
Jakeman, A.J., Letcher, R.A., Norton, J.P., Ten iterative steps in development and evaluation of environmental models. Environ. Model. Software 21 (2006), 602–614, 10.1016/j.envsoft.2006.01.004.
Jansen, M.J.W., Hagenaars, T.J., Calibration in a Bayesian modelling framework. van Boekel, M.A.J.S., Stein, A., van Bruggen, A.H.C., (eds.) Bayesian Statistics and Quality Modelling in the Agro-Food Production Chain, 2004, Kluwer Academic, Dordrecht.
Kawakita, S., Takahashi, H., Moriya, K., Prediction and parameter uncertainty for winter wheat phenology models depend on model and parameterization method differences. Agric. For. Meteorol., 2020, 107998, 10.1016/J.AGRFORMET.2020.107998.
Keating, B.A., Thorburn, P.J., Modelling crops and cropping systems—evolving purpose, practice and prospects. Eur. J. Agron. 100 (2018), 163–176, 10.1016/j.eja.2018.04.007.
Kimball, B.A., Boote, K.J., Hatfield, J.L., Ahuja, L.R., Stockle, C., Archontoulis, S., Baron, C., Basso, B., Bertuzzi, P., Constantin, J., Deryng, D., Dumont, B., Durand, J.-L., Ewert, F., Gaiser, T., Gayler, S., Hoffmann, M.P., Jiang, Q., Kim, S.-H., Lizaso, J., Moulin, S., Nendel, C., Parker, P., Palosuo, T., Priesack, E., Qi, Z., Srivastava, A., Stella, T., Tao, F., Thorp, K.R., Timlin, D., Twine, T.E., Webber, H., Willaume, M., Williams, K., Simulation of maize evapotranspiration: an inter-comparison among 29 maize models. Agric. For. Meteorol. 271 (2019), 264–284, 10.1016/J.AGRFORMET.2019.02.037.
Li, T., Hasegawa, T., Yin, X., Zhu, Y., Boote, K., Adam, M., Bregaglio, S., Buis, S., Confalonieri, R., Fumoto, T., Gaydon, D., Marcaida, M., Nakagawa, H., Oriol, P., Ruane, A.C., Ruget, F., Singh, B., Singh, U., Tang, L., Tao, F., Wilkens, P., Yoshida, H., Zhang, Z., Bouman, B., Uncertainties in predicting rice yield by current crop models under a wide range of climatic conditions. Global Change Biol. 21 (2015), 1328–1341, 10.1111/gcb.12758.
Liu, L., Wallach, D., Li, J., Liu, B., Zhang, L., Tang, L., Zhang, Y., Qiu, X., Cao, W., Zhu, Y., Uncertainty in wheat phenology simulation induced by cultivar parameterization under climate warming. Eur. J. Agron. 94 (2018), 46–53, 10.1016/J.EJA.2017.12.001.
Madsen, H., Wilson, G., Ammentorp, H.C., Comparison of different automated strategies for calibration of rainfall-runoff models. J. Hydrol. 261 (2002), 48–59.
Meyer Oliveira, A., Fleischmann, A.S., Paiva, R.C.D., On the contribution of remote sensing-based calibration to model hydrological and hydraulic processes in tropical regions. J. Hydrol., 126184, 2021, 10.1016/j.jhydrol.2021.126184.
Necpálová, M., Anex, R.P., Fienen, M.N., Del Grosso, S.J., Castellano, M.J., Sawyer, J.E., Iqbal, J., Pantoja, J.L., Barker, D.W., Understanding the DayCent model: calibration, sensitivity, and identifiability through inverse modeling. Environ. Model. Software 66 (2015), 110–130, 10.1016/J.ENVSOFT.2014.12.011.
Nelder, J.A., Mead, R., A simplex method for function minimization. Comput. J. 7 (1965), 308–313, 10.1093/comjnl/7.4.308.
NIST/SEMATECH. E-Handbook of Statistical Methods. 2013, 10.18434/M32189.
Piao, S., Liu, Q., Chen, A., Janssens, I.A., Fu, Y., Dai, J., Liu, L., Lian, X., Shen, M., Zhu, X., Plant phenology and global climate change: current progresses and challenges. Glob. Chang. Biol. gcb., 14619, 2019, 10.1111/gcb.14619.
Rezaei, E.E., Siebert, S., Ewert, F., Intensity of heat stress in winter wheat—phenology compensates for the adverse effect of global warming. Environ. Res. Lett., 10, 2015, 024012, 10.1088/1748-9326/10/2/024012.
Rezaei, E.E., Siebert, S., Hüging, H., Ewert, F., Climate change effect on wheat phenology depends on cultivar change. Sci. Rep., 8, 2018, 4891, 10.1038/s41598-018-23101-2.
Rosenzweig, C., Jones, J.W., Hatfield, J.L., Ruane, A.C., Boote, K.J., Thorburn, P., Antle, J.M., Nelson, G.C., Porter, C., Janssen, S., Asseng, S., Basso, B., Ewert, F., Wallach, D., Baigorria, G., Winter, J.M., The agricultural model Intercomparison and improvement project (AgMIP): protocols and pilot studies. Agric. For. Meteorol. 170 (2013), 166–182, 10.1016/j.agrformet.2012.09.011.
Seber, G.A.F., Wild, C.J., Nonlinear Regression. 1989, Wiley, New York.
Seidel, S.J., Palosuo, T., Thorburn, P., Wallach, D., Towards improved calibration of crop models – where are we now and where should we go?. Eur. J. Agron. 94 (2018), 25–35, 10.1016/J.EJA.2018.01.006.
Sen, A., Srivastava, M., Regression Analysis : Theory, Methods, and Applications. 1990, Springer, New York.
Sexton, J., Everingham, Y., Inman-Bamber, G., A theoretical and real world evaluation of two Bayesian techniques for the calibration of variety parameters in a sugarcane crop model. Environ. Model. Software 83 (2016), 126–142, 10.1016/j.envsoft.2016.05.014.
Skahill, B.E., Doherty, J., Efficient accommodation of local minima in watershed model calibration. J. Hydrol. 329 (2006), 122–139, 10.1016/J.JHYDROL.2006.02.005.
Steele, K., Werndl, C., Climate models, calibration, and confirmation. Br. J. Philos. Sci. 64 (2013), 609–635, 10.1093/bjps/axs036.
Tsuji, G.Y., Hoogenboom, G., Thornton, P.K., (eds.) Understanding Options for Agricultural Production, Systems Approaches for Sustainable Agricultural Development, 1998, Springer Netherlands, Dordrecht, 10.1007/978-94-017-3624-4.
Van Oijen, M., Rougier, J., Smith, R., Bayesian calibration of process-based forest models: bridging the gap between models and data. Tree Physiol. 25 (2005), 915–927.
Wallach, D., Crop model calibration: a statistical perspective. Agron. J. 103 (2011), 1144–1151.
Wallach, D., Hwang, C., Correll, M.J., Jones, J.W., Boote, K., Hoogenboom, G., Gezan, S., Bhakta, M., Vallejos, C.E., A dynamic model with QTL covariables for predicting flowering time of common bean (Phaseolus vulgaris) genotypes. Eur. J. Agron. 101 (2018), 200–209, 10.1016/J.EJA.2018.10.003.
Wallach, D., Makowski, D., Jones, J.W., Brun, F., Working with Dynamic Crop Models : Methods, Tools and Examples for Agriculture and Environment. Third. ed., 2019, Academic Press, London, U.K.
Wallach, D., Palosuo, T., Thorburn, P., Gourdain, E., Asseng, S., Basso, B., Buis, S., Crout, N., Dibari, C., Dumont, B., Ferrise, R., Gaiser, T., Garcia, C., Gayler, S., Ghahramani, A., Hochman, Z., Hoek, S., Hoogenboom, G., Horan, H., Huang, M., Jabloun, M., Jing, Q., Justes, E., Kersebaum, K.C., Klosterhalfen, A., Launay, M., Luo, Q., Maestrini, B., Mielenz, H., Moriondo, M., Nariman Zadeh, H., Olesen, J.E., Poyda, A., Priesack, E., Pullens, J.W.M., Qian, B., Schütze, N., Shelia, V., Souissi, A., Specka, X., Srivastava, A.K., Stella, T., Streck, T., Trombi, G., Wallor, E., Wang, J., Weber, T.K.D., Weihermüller, L., de Wit, A., Wöhling, T., Xiao, L., Zhao, C., Zhu, Y., Seidel, S.J., How well do crop modeling groups predict wheat phenology, given calibration data from the target population?. Eur. J. Agron., 124, 2021, 126195, 10.1016/j.eja.2020.126195.
Wallach, D., Palosuo, T., Thorburn, P., Hochman, Z., Andrianasolo, F., Asseng, S., Basso, B., Buis, S., Crout, N., Dumont, B., Ferrise, R., Gaiser, T., Gayler, S., Hiremath, S., Hoek, S., Horan, H., Hoogenboom, G., Huang, M., Jabloun, M., Jansson, P.-E., Jing, Q., Justes, E., Kersebaum, K.C., Launay, M., Lewan, E., Luo, Q., Maestrini, B., Moriondo, M., Padovan, G., Olesen, J.E., Poyda, A., Priesack, E., Pullens, J.W.M., Qian, B., Schütze, N., Shelia, V., Souissi, A., Specka, X., Srivastava, A.K., Stella, T., Streck, T., Trombi, G., Wallor, E., Wang, J., Weber, T.K.D., Weihermüller, L., Wit, A. de, Wöhling, T., Xiao, L., Zhao, C., Zhu, Y., Seidel, S.J., Multi model evaluation of phenology prediction for wheat in Australia. Agric. For. Meteorol, 2021, 10.1101/2020.06.06.133504 (in press).
Wang, X., Kemanian, A., Williams, J., Special features of the EPIC and APEX modeling package and procedures for parameterization, calibration, validation, and applications. Ahuja, L.R., Ma, L., (eds.) Methods of Introducing System Models into Agricultural Research, 2011, American Society of Agronomy, Madison, 177–208.
Willmott, C.J., Matsuura, K., Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Clim. Res. 30 (2005), 79–82.
Wöhling, T., Gayler, S., Priesack, E., Ingwersen, J., Wizemann, H.-D., Högy, P., Cuntz, M., Attinger, S., Wulfmeyer, V., Streck, T., Multiresponse, multiobjective calibration as a diagnostic tool to compare accuracy and structural limitations of five coupled soil-plant models and CLM3.5. Water Resour. Res. 49 (2013), 8200–8221, 10.1002/2013WR014536.
Wöhling, T., Samaniego, L., Kumar, R., Evaluating multiple performance criteria to calibrate the distributed hydrological model of the upper Neckar catchment. Environ. Earth Sci. 69 (2013), 453–468, 10.1007/s12665-013-2306-2.
Wu, L., Feng, L., Zhang, Y., Gao, J., Wang, J., Comparison of five wheat models simulating phenology under different sowing dates and varieties. Agron. J., 109, 2017, 10.2134/agronj2016.10.0619.
Zadoks, J.C., Chzang, T.T., Konzak, C.F., A decimal code for the growth stages of cereals. Weed Res. 14 (1974), 415–421, 10.1111/j.1365-3180.1974.tb01084.x.
Zhang, S., Tao, F., Zhang, Z., Uncertainty from model structure is larger than that from model parameters in simulating rice phenology in China. Eur. J. Agron. 87 (2017), 30–39, 10.1016/j.eja.2017.04.004.