Poster (Scientific congresses and symposiums)
SAE: Sequential Anchored Ensembles
Delaunoy, Arnaud; Louppe, Gilles
2021Bayesian Deep Learning, NeurIPS 2021 workshop
Peer reviewed
 

Files


Full Text
2201.00649.pdf
Author postprint (245.2 kB)
Download

All documents in ORBi are protected by a user license.

Send to



Details



Keywords :
Computer Science - Learning
Abstract :
[en] Computing the Bayesian posterior of a neural network is a challenging task due to the high-dimensionality of the parameter space. Anchored ensembles approximate the posterior by training an ensemble of neural networks on anchored losses designed for the optima to follow the Bayesian posterior. Training an ensemble, however, becomes computationally expensive as its number of members grows since the full training procedure is repeated for each member. In this note, we present Sequential Anchored Ensembles (SAE), a lightweight alternative to anchored ensembles. Instead of training each member of the ensemble from scratch, the members are trained sequentially on losses sampled with high auto-correlation, hence enabling fast convergence of the neural networks and efficient approximation of the Bayesian posterior. SAE outperform anchored ensembles, for a given computational budget, on some benchmarks while showing comparable performance on the others and achieved 2nd and 3rd place in the light and extended tracks of the NeurIPS 2021 Approximate Inference in Bayesian Deep Learning competition.
Disciplines :
Computer science
Mathematics
Author, co-author :
Delaunoy, Arnaud ;  Université de Liège - ULiège > Montefiore Institute of Electrical Engineering and Computer Science
Louppe, Gilles  ;  Université de Liège - ULiège > Département d'électricité, électronique et informatique (Institut Montefiore) > Big Data
Language :
English
Title :
SAE: Sequential Anchored Ensembles
Publication date :
14 December 2021
Event name :
Bayesian Deep Learning, NeurIPS 2021 workshop
Event date :
December 14, 2021
Audience :
International
Peer reviewed :
Peer reviewed
Funders :
F.R.S.-FNRS - Fonds de la Recherche Scientifique
NRB
Funding text :
The authors would like to thank Tim Pearce for his insightful comments and feedback. Arnaud Delaunoy would like to thank the National Fund for Scientific Research (F.R.S.-FNRS) for his scholarship. Gilles Louppe is recipient of the ULiège - NRB Chair on Big Data and is thankful for the support of the NRB.
Commentary :
4 pages, NeurIPS 2021 Bayesian Deep Learning workshop
Available on ORBi :
since 18 May 2022

Statistics


Number of views
103 (15 by ULiège)
Number of downloads
13 (0 by ULiège)

Bibliography


Similar publications



Contact ORBi