Sensitivity analysis of prior model probabilities and the value of prior knowledge in the assessment of conceptual model uncertainty in groundwater modelling
Rojas, Rodrigo; Feyen, Luc; Dassargues, Alain
2009 • In Hydrological Processes, 23, p. 1131-1146
Groundwater modelling; conceptual model uncertainty; prior knowledge; maximum entropy; uncertainty assessment; multi-model prediction
Abstract :
[en] A key point in the application of multi-model Bayesian averaging techniques to assess the predictive uncertainty in groundwater modelling applications is the definition of prior model probabilities, which reflect the prior perception about the plausibility of alternative models. In this work the influence of prior knowledge and prior model probabilities on posterior model probabilities, multi-model predictions, and conceptual model uncertainty estimations is analysed. The sensitivity to prior model probabilities is assessed using an extensive numerical analysis in which the prior probability space of a set of plausible conceptualizations is discretized to obtain a large ensemble of possible combinations of prior model probabilities. Additionally, the value of prior knowledge about alternative models in reducing conceptual model uncertainty is assessed by considering three example knowledge states, expressed as quantitative relations among the alternative models. A constrained maximum entropy approach is used to find the set of prior model probabilities that correspond to the different prior knowledge states. For illustrative purposes, a three-dimensional hypothetical setup approximated by seven alternative conceptual models is employed. Results show that posterior model probabilities, leading moments of the predictive distributions and estimations of conceptual model uncertainty are very sensitive to prior model probabilities, indicating the relevance of selecting proper prior probabilities. Additionally, including proper prior knowledge improves the predictive performance of the multi-model approach, expressed by reductions of the multi-model prediction variances by up to 60% compared with a non-informative case. However, the ratio between-model to total variance does not substantially decrease. This suggests that the contribution of conceptual model uncertainty to the total variance cannot be further reduced based only on prior knowledge about the plausibility of alternative models. These results advocate including proper prior knowledge about alternative conceptualizations in combination with extra conditioning data to further reduce conceptual model uncertainty in groundwater modelling predictions.
Research Center/Unit :
Aquapôle - ULiège
Disciplines :
Geological, petroleum & mining engineering
Author, co-author :
Rojas, Rodrigo; Katholieke Universiteit Leuven - KUL > Department of Earth and Environmental Sciences > Hydrogeology, Applied Mineralogy and Geology
Feyen, Luc; DG Joint Research Centre, European Commission, Ispra > Institute for Environment and Sustainability > Land management and natural hazards unit
Dassargues, Alain ; Université de Liège - ULiège > Département Argenco : Secteur GEO3 > Hydrogéologie & Géologie de l'environnement
Language :
English
Title :
Sensitivity analysis of prior model probabilities and the value of prior knowledge in the assessment of conceptual model uncertainty in groundwater modelling
Publication date :
January 2009
Journal title :
Hydrological Processes
ISSN :
0885-6087
eISSN :
1099-1085
Publisher :
John Wiley & Sons, Inc, Chichester, United Kingdom
Ajami N, Duan Q, Gao X, Sorooshian S. 2005. Multi-model combination techniques for hydrologic forecasting: application to distributed model intercomparison project results. Journal of Hydrometeorology 7(4): 755-768, DOI:10.1175/JHM519.1.
Ajami N, Duan Q, Sorooshian S. 2007. An integrated hydrologic Bayesian multi-model combination framework: Confronting input, parameter, and model structural uncertainty in hydrologic prediction. Water Resources Research 43: DOI:10.1029/2005WR004745.
Applebaum D. 1996. Probability and information: an integrated approach, 1st edn. Cambridge University Press: Cambridge.
Beven K. 2006. A manifesto for the equifinality thesis. Journal of Hydrology 320(1-2): 18-36, DOI:10.1016/j.jhydrol.2005.07.007.
Beven K, Binley A. 1992. The future of distributed models: model calibration and uncertainty prediction. Hydrological Processes 6(5): 279-283, DOI:10.1002/hyp.3360060305.
Beven K, Freer J. 2001. Equifinality, data assimilation, and uncertainty estimation in mechanistic modelling of complex environmental systems using the GLUE methodology. Journal of Hydrology 249(1-4): 11-29, DOI:10.1016/S0022-1694(01)00421-8.
Beven K, Smith P, Freer J. 2008. So just why would a modeller choose to be incoherent? Journal of Hydrology 354(1-4): 15-32, DOI:10.1016/j.jhydrol.2008.02.007.
Binley A, Beven K. 2003. Vadose zone flow model uncertainty as conditioned on geophysical data. Ground Water 41(2): 119-127, DOI:10.1111/j.1745-6584.2003.tb02576.x.
Bredehoeft J. 2003. From models to performance assessment: the conceptualization problem. Ground Water 41(5): 571-577, DOI:10.1111/j.1745-6584.2003.tb02395.x.
Bredehoeft J. 2005. The conceptualization model problem - surprise. Hydrogeology Journal 13(1): 37-46, DOI:10.1007/ s10040-004-0430-5.
Carrera J, Alcolea A, Medina A, Hidalgo J, Slooten L. 2005. Inverse problem in hydrogeology. Hydrogeology Journal 13(1): 206-222, DOI:10.1007/s10040-004-0404-7.
Deutsch C, Journel A. 1998. GSLIB: Geostatistical Software Library and User's Guide, 2nd edn. Oxford University Press: New York.
Draper D. 1995. Assessment and propagation of model uncertainty. Journal of the Royal Statistical Society Series B 57(1): 45-97.
Gelman A, Carlin J, Stern H, Rubin D. 2004. Bayesian Data Analysis, 2nd edn. Chapman & Hall/CRC: Boca Raton.
Ghosh J, Delampady M, Samanta T. 2006. An Introduction to Bayesian Analysis - Theory and Methods, 1st edn. Springer-Verlag: New York.
Gilks W, Richardson S, Spiegelhalter D. 1995. Markov Chain Monte Carlo in Practice, 1st edn. Chapman & Hall/CRC: Boca Raton.
Harbaugh A, Banta E, Hill M, McDonald M. 2000. MODFLOW-2000 US Geological Survey modular ground-water model-user guide to modularization concepts and the ground-water flow process. Open File Rep., 00-92, US Geological Survey.
Harrar W, Sonnenberg T, Henriksen H. 2003. Capture zone, travel time, and solute transport predictions using inverse modelling and different geological models. Hydrogeology Journal 11(5): 536-548, DOI:10.1007/s10040-003-0276-2.
Hoeting J, Madigan D, Raftery A, Volinsky C. 1999. Bayesian model averaging: a tutorial. Statistical Science 14(4): 382-417.
Højberg A, Refsgaard JC. 2005. Model uncertainty - parameter uncertainty versus conceptual models. Water Science & Technology 52(6): 177-186.
Kass R, Raftery A. 1995. Bayes factors. Journal of the American Statistical Association 90(430): 773-795.
Kass R, Wasserman L. 1996. The selection of prior distributions by formal rules. Journal of the American Statistical Association 91(435): 1343-1370.
Liang F, Truong Y, Wong W. 2001. Automatic Bayesian model averaging for linear regression and application in Bayesian curve fitting. Statistica Sinica 11(4): 1005-1029.
McKay D, Beckman R, Conover W. 1979. A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2): 239-245.
Meyer P, Ye M, Rockhold M, Neuman S, Cantrell K. 2007. Combined estimation of hydrogeologic conceptual model parameter and scenario uncertainty with application to uranium transport at the Hanford Site 300 area. Report No. NUREG/CR-6940 PNNL-16396, US Nuclear Regulatory Commission.
Montanari A. 2005. Large sample behaviors of the generalized likelihood uncertainty estimation (GLUE) in assessing the uncertainty of rainfall-runoff simulations. Water Resources Research 41: DOI:10.1029/2004WR003826.
Neuman S. 2003. Maximum likelihood Bayesian averaging of uncertain model predictions. Stochastic Environmental Research and Risk Assessment 17(5): 291-305, DOI:10.1007/s00477-003-0151-7.
Neuman S, Wierenga P. 2003. A comprehensive strategy of hydrogeologic modelling and uncertainty analysis for nuclear facilities and sites. Report No. NUREG/CR-6805, US Nuclear Regulatory Commission.
Poeter E, Anderson D. 2005. Multi-model ranking and inference in ground water modelling. Ground Water 43(4): 597-605, DOI:10.1111/ j.1745-6584.2005.0061.x.
Raftery A, Zhang Y. 2003. Discussion: performance of Bayesian model averaging. Journal of the American Statistical Association 98(464): 931-938.
Refsgaard JC, van der Sluijs J, Brown J, van der Keur P. 2006. A framework for dealing with uncertainty due to model structure error. Advances in Water Resources 29(11): 1586-1897, DOI:10.1016/ j.advwatres.2005.11.013.
Rojas R, Feyen L, Dassargues A. 2008. Conceptual model uncertainty in groundwater modelling: Combining generalized likelihood uncertainty estimation and Bayesian model averaging. Water Resources Research 44: W12418. DOI:10.1029/2008WR006908.
Romanowicz R, Beven K, Tawn J. 1994. Evaluation of prediction uncertainty in non-linear hydrological models using a Bayesian approach. In Statistics for the Environment 2 - Water Related Issues, Barnett V, Trukman F (eds). John Wiley & Sons: Chichester; 297-317.
Rubin Y. 2003. Applied Stochastic Hydrogeology, 1st edn. Oxford University Press: New York.
Shannon C. 1948. A mathematical theory of communication. Bell System Technical Journal 27(279-423): 623-656.
Spellucci P. 1998. An SQP method for general nonlinear programs using only equality constrained subproblems. Mathematical Programming 82(3): 413-448, DOI:10.1007/BF01580078.
Tamura R. 2007. Rdonlp2: An R extension library to use Peter Spellucci's DONLP2 from R. R package version 0.3-1. R package version 0.3-1 [On-line]. Available: http://arumat.net/Rdonlp2/.
Wasserman L. 2000. Bayesian model selection and model averaging. Journal of Mathematical Psychology 44(1): 92-107, DOI:10.1006/jmps.1999.1278.
Ye M, Neuman S, Meyer P. 2004. Maximum likelihood Bayesian averaging of spatial variability models in unsaturated fractured tuff. Water Resources Research 40: DOI:10.1029/2003WR002557.
Ye M, Neuman S, Meyer P, Pohlmann K. 2005. Sensitivity analysis and assessment of prior model probabilities in MLBMA with application to unsaturated fractured tuff. Water Resources Research 41: DOI:10.1029/2005WR004260.
Ye M, Pohlmann K, Chapman J, Shafer D. 2006. On evaluation of recharge model uncertainty: a priori and a posteriori. In: Proceedings of the International High-Level Radioactive Waste Management Conference. Las Vegas, Nevada U.S.; 12.
Ye M, Pohlmann K, Chapman J. 2008. Expert elicitation of recharge model probabilities for the Death Valley regional flow system. Journal of Hydrology 354(1-4): 102-115, DOI:10.1016/ j.jhydrol.2008.03.001.