Statistics - Machine Learning; Computer Science - Learning; Computer Science - Neural and Evolutionary Computing; Physics - Data Analysis; Statistics and Probability; Statistics - Methodology
Abstract :
[en] Several techniques for domain adaptation have been proposed to account for differences in the distribution of the data used for training and testing. The majority of this work focuses on a binary domain label. Similar problems occur in a scientific context where there may be a continuous family of plausible data generation processes associated to the presence of systematic uncertainties. Robust inference is possible if it is based on a pivot -- a quantity whose distribution does not depend on the unknown values of the nuisance parameters that parametrize this family of data generation processes. In this work, we introduce and derive theoretical results for a training procedure based on adversarial networks for enforcing the pivotal property (or, equivalently, fairness with respect to continuous attributes) on a predictive model. The method includes a hyperparameter to control the trade-off between accuracy and robustness. We demonstrate the effectiveness of this approach with a toy example and examples from particle physics.
Disciplines :
Computer science
Author, co-author :
Louppe, Gilles ; Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Big Data
Kagan, Michael
Cranmer, Kyle
Language :
English
Title :
Learning to Pivot with Adversarial Networks
Publication date :
November 2016
Event name :
Neural Information Processing Systems Conference (NIPS) 2017
Event place :
Long Beach, United States
Event date :
December 2017
Audience :
International
Journal title :
Advances in Neural Information Processing Systems
ISSN :
1049-5258
Publisher :
Morgan Kaufmann Publishers, San Mateo, United States - California
Adam-Bourdarios, C., Cowan, G., Germain, C., Guyon, I., Kégl, B., and Rousseau, D. (2014). The higgs boson machine learning challenge. In NIPS 2014 Workshop on High-energy Physics and Machine Learning, Volume 42, page 37.
Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., and Marchand, M. (2014). Domain-adversarial neural networks. arXiv preprint arXiv:1412.4446.
ATLAS Collaboration (2014). Performance of Boosted W Boson Identification with the ATLAS Detector. Technical Report ATL-PHYS-PUB-2014-004, CERN, Geneva.
ATLAS Collaboration (2015). Identification of boosted, hadronically-decaying W and Z bosons in √s = 13 TeV Monte Carlo Simulations for ATLAS. Technical Report ATL-PHYS-PUB-2015-033, CERN, Geneva.
Baktashmotlagh, M., Harandi, M., Lovell, B., and Salzmann, M. (2013). Unsupervised domain adaptation by domain invariant projection. In Proceedings of the IEEE International Conference on Computer Vision, pages 769-776.
Baldi, P., Bauer, K., Eng, C., Sadowski, P., and Whiteson, D. (2016a). Jet substructure classification in high-energy physics with deep neural networks. Physical Review D, 93(9):094034.
Baldi, P., Cranmer, K., Faucett, T., Sadowski, P., and Whiteson, D. (2016b). Parameterized neural networks for high-energy physics. Eur. Phys. J., C76(5):235.
Bishop, C. M. (1994). Mixture density networks.
Blitzer, J., McDonald, R., and Pereira, F. (2006). Domain adaptation with structural correspondence learning. In Proceedings of the 2006 conference on empirical methods in natural language processing, pages 120-128. Association for Computational Linguistics.
CMS Collaboration (2014). Identification techniques for highly boosted W bosons that decay into hadrons. JHEP, 12:017.
Cranmer, K., Pavez, J., and Louppe, G. (2015). Approximating likelihood ratios with calibrated discriminative classifiers. arXiv preprint arXiv:1506.02169.
Degroot, M. H. and Schervish, M. J. (1975). Probability and statistics. 1st edition.
Edwards, H. and Storkey, A. J. (2015). Censoring representations with an adversary. arXiv preprint arXiv:1511.05897.
Evans, L. and Bryant, P. (2008). LHC Machine. JINST, 3:S08001.
Feldman, M., and Friedler, S. A., Moeller, J., Scheidegger, C., and Venkatasubramanian, S. (2015). Certifying and removing disparate impact. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 259-268. ACM.
Ganin, Y. and Lempitsky, V. (2014). Unsupervised Domain Adaptation by Backpropagation. arXiv preprint arXiv:1409.7495.
Gong, B., Grauman, K., and Sha, F. (2013). Connecting the dots with landmarks: Discriminatively learning domain-invariant features for unsupervised domain adaptation. In Proceedings of The 30th International Conference on Machine Learning, pages 222-230.
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative adversarial nets. In Advances in Neural Information Processing Systems, pages 2672-2680.
Gopalan, R., Li, R., and Chellappa, R. (2011). Domain adaptation for object recognition: An unsupervised approach. In Computer Vision (ICCV), 2011 IEEE International Conference on, pages 999-1006. IEEE.
Kamishima, T., Akaho, S., Asoh, H., and Sakuma, J. (2012). Fairness-aware classifier with prejudice remover regularizer. Machine Learning and Knowledge Discovery in Databases, pages 35-50.
Louizos, C., Swersky, K., Li, Y., Welling, M., and Zemel, R. (2015). The variational fair autoencoder. arXiv preprint arXiv:1511.00830.
Pan, S. J., and Tsang, I. W., Kwok, J. T., and Yang, Q. (2011). Domain adaptation via transfer component analysis. Neural Networks, IEEE Transactions on, 22(2):199-210.
Shimmin, C., Sadowski, P., Baldi, P., Weik, E., Whiteson, D., Goul, E., and Søgaard, A. (2017). Decorrelated Jet Substructure Tagging using Adversarial Neural Networks.
Zafar, M. B., Valera, I., Rodriguez, M. G., and Gummadi, K. P. (2015). Fairness constraints: A mechanism for fair classification. arXiv preprint arXiv:1507.05259.
Zemel, R. S., Wu, Y., Swersky, K., Pitassi, T., and Dwork, C. (2013). Learning fair representations. ICML (3), 28:325-333.
