Article (Scientific journals)
A Crisis In Simulation-Based Inference? Beware, Your Posterior Approximations Can Be Unfaithful
Hermans, Joeri; Delaunoy, Arnaud; Rozet, François et al.
2022In Transactions on Machine Learning Research
Peer Reviewed verified by ORBi
 

Files


Full Text
2110.06581.pdf
Author preprint (749.9 kB)
Download

All documents in ORBi are protected by a user license.

Send to



Details



Keywords :
Simulation-based inference; Reliable inference; Likelihood-free inference; Conservative inference; Bayesian inference; Machine learning
Abstract :
[en] We present extensive empirical evidence showing that current Bayesian simulation-based inference algorithms are inadequate for the falsificationist methodology of scientific inquiry. Our results collected through months of experimental computations show that all benchmarked algorithms - (s)npe, (s)nre, snl and variants of abc – may produce overconfident posterior approximations, which makes them demonstrably unreliable and dangerous if one’s scientific goal is to constrain parameters of interest. We believe that failing to address this issue will lead to a well-founded trust crisis in simulation-based inference. For this reason, we argue that research efforts should now consider theoretical and methodological developments of conservative approximate inference algorithms and present research directions towards this objective. In this regard, we show empirical evidence that ensembles are consistently more reliable.
Disciplines :
Computer science
Author, co-author :
Hermans, Joeri ;  Université de Liège - ULiège > Montefiore Institute
Delaunoy, Arnaud ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Big Data
Rozet, François  ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Big Data
Wehenkel, Antoine  ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Big Data
Louppe, Gilles  ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Big Data
Language :
English
Title :
A Crisis In Simulation-Based Inference? Beware, Your Posterior Approximations Can Be Unfaithful
Publication date :
November 2022
Journal title :
Transactions on Machine Learning Research
eISSN :
2835-8856
Publisher :
OpenReview, Amherst, United States - Massachusetts
Peer reviewed :
Peer Reviewed verified by ORBi
Tags :
CÉCI : Consortium des Équipements de Calcul Intensif
Funders :
F.R.S.-FNRS - Fonds de la Recherche Scientifique [BE]
CÉCI - Consortium des Équipements de Calcul Intensif [BE]
NRB
Available on ORBi :
since 19 November 2021

Statistics


Number of views
157 (22 by ULiège)
Number of downloads
93 (9 by ULiège)

Bibliography


Similar publications



Contact ORBi