deep learning; geophysical inversion; ground-penetrating radar; prior information; traveltime tomography; variational autoencoder; Geophysics; Geochemistry and Petrology; Space and Planetary Science; Earth and Planetary Sciences (miscellaneous)
Abstract :
[en] Prior information regarding subsurface spatial patterns may be used in geophysical inversion to obtain realistic subsurface models. Field experiments require prior information with sufficiently diverse patterns to accurately estimate the spatial distribution of geophysical properties in the sensed subsurface domain. A variational autoencoder (VAE) provides a way to assemble all patterns deemed possible in a single prior distribution. Such patterns may include those defined by different base training images and also their perturbed versions, for example, those resulting from geologically consistent operations such as erosion/dilation, local deformation, and intrafacies variability. Once the VAE is trained, inversion may be done in the latent space which ensures that inverted models have the patterns defined by the assembled prior. Gradient-based inversion with both a synthetic and a field case of cross-borehole GPR traveltime data shows that using the VAE assembled prior performs as good as using the VAE trained on the pattern with the best fit, but it has the advantage of lower computation cost and more realistic prior uncertainty. Moreover, the synthetic case shows an adequate estimation of most small-scale structures. The absolute values of wave velocity are computed by assuming a linear mixing model which involves two additional parameters that effectively shift and scale velocity values and are included in the inversion.
Disciplines :
Geological, petroleum & mining engineering
Author, co-author :
Lopez-Alvis, J. ; Urban and Environmental Engineering, Applied Geophysics, University of Liege, Liege, Belgium
Nguyen, Frédéric ; Université de Liège - ULiège > Département ArGEnCo > Géophysique appliquée
Looms, M.C. ; Department of Geosciences and Natural Resource Management, University of Copenhagen, Copenhagen, Denmark
Hermans, T. ; Department of Geology, Ghent University, Gent, Belgium
Language :
English
Title :
Geophysical Inversion Using a Variational Autoencoder to Model an Assembled Spatial Prior Uncertainty
H2020 - 722028 - ENIGMA - European training Network for In situ imaGing of dynaMic processes in heterogeneous subsurfAce environments
Funders :
EU - European Union [BE]
Funding text :
This work has received funding from the European Unions Horizon 2020 research and innovation program under the Marie Sklodowska‐Curie grant agreement number 722 028 (ENIGMA ITN). We thank two anonymous reviewers and the associate editor for their valuable comments that greatly improved the manuscript.
Backus, G. E., & Gilbert, J. F. (1967). Numerical applications of a formalism for geophysical inverse problems. Geophysical Journal of the Royal Astronomical Society, 13(1–3), 247–276. https://doi.org/10.1111/j.1365-246X.1967.tb02159.x
Bergmann, U., Jetchev, N., & Vollgraf, R. (2017). Learning Texture manifolds with the Periodic spatial GAN. ArXiv:1705.06566 [cs, stat]. Retrieved from http://arxiv.org/abs/1705.06566
Bora, A., Jalal, A., Price, E., & Dimakis, A. G. (2017). Compressed sensing using generative models. ArXiv:1703.03208 [cs, math, stat]. Retrieved from http://arxiv.org/abs/1703.03208
Bording, T. S., Fiandaca, G., Maurya, P. K., Auken, E., Christiansen, A. V., Tuxen, N., & Larsen, T. H. (2019). Cross-borehole tomography with full-decay spectral time-domain induced polarization for mapping of potential contaminant flow-paths. Journal of Contaminant Hydrology, 226, 103523. https://doi.org/10.1016/j.jconhyd.2019.103523
Caers, J., & Hoffman, T. (2006). The probability perturbation method: A new look at Bayesian inverse modeling. Mathematical Geology, 38(1), 81–100. https://doi.org/10.1007/s11004-005-9005-9
Canchumuni, S. W., Emerick, A. A., & Pacheco, M. A. C. (2019). Towards a robust parameterization for conditioning facies models using deep variational autoencoders and ensemble smoother. Computers & Geosciences, 128, 87–102. https://doi.org/10.1016/j.cageo.2019.04.006
Chan, S., & Elsheikh, A. H. (2019). Parametrization and generation of geological models with generative adversarial networks. ArXiv:1708.01810 [physics, stat]. Retrieved from http://arxiv.org/abs/1708.01810
Davis, K., & Li, Y. (2011). Fast solution of geophysical inversion using adaptive mesh, space-filling curves and wavelet compression. Geophysical Journal International, 185(1), 157–166. https://doi.org/10.1111/j.1365-246X.2011.04929.x
Dayan, P., Hinton, G. E., Neal, R. M., & Zemel, R. S. (1995). The Helmholtz machine. Neural Computation, 7(5), 889–904. https://doi.org/10.1162/neco.1995.7.5.889
Day-Lewis, F. D., Singha, K., & Binley, A. M. (2005). Applying petrophysical models to radar travel time and electrical resistivity tomograms: Resolution-dependent limitations. Journal of Geophysical Research, 110, B08206. https://doi.org/10.1029/2004JB003569
Fefferman, C., Mitter, S., & Narayanan, H. (2016). Testing the manifold hypothesis. Journal of the American Mathematical Society, 29(4), 983–1049. https://doi.org/10.1090/jams/852
Franklin, J. N. (1970). Well-posed stochastic extensions of ill-posed linear problems. Journal of Mathematical Analysis and Applications, 31(3), 682–716.
Fukushima, K. (1980). Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36(4), 193–202. https://doi.org/10.1007/BF00344251
Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., et al. (2014). Generative adversarial networks. ArXiv:1406.2661 [cs, stat]. Retrieved from http://arxiv.org/abs/1406.2661
Hansen, T. M., Cordua, K. S., Jacobsen, B. H., & Mosegaard, K. (2014). Accounting for imperfect forward modeling in geophysical inverse problems—Exemplified for crosshole tomography. Geophysics, 79(3), H1–H21. https://doi.org/10.1190/geo2013-0215.1
Hansen, T. M., Cordua, K. S., & Mosegaard, K. (2012). Inverse problems with non-trivial priors: Efficient solution through sequential Gibbs sampling. Computational Geosciences, 16(3), 593–611. https://doi.org/10.1007/s10596-011-9271-1
Hermans, T., Nguyen, F., & Caers, J. (2015). Uncertainty in training image-based inversion of hydraulic head data constrained to ERT data: Workflow and case study. Water Resources Research, 51, 5332–5352. https://doi.org/10.1002/2014WR016460
Hermans, T., Vandenbohede, A., Lebbe, L., Martin, R., Kemna, A., Beaujean, J., & Nguyen, F. (2012). Imaging artificial salt water infiltration using electrical resistivity tomography constrained by geostatistical data. Journal of Hydrology, 438–439, 168–180. https://doi.org/10.1016/j.jhydrol.2012.03.021
Higgins, I., Matthey, L., Pal, A., Burgess, C., Glorot, X., Botvinick, M., et al. (2017). Beta-VAE: Learning basic visual concepts with a constrained variational framework.
Holm-Jensen, T., & Hansen, T. M. (2019). Linear Waveform tomography inversion using machine learning algorithms. Mathematical Geosciences, 52, 31–51. https://doi.org/10.1007/s11004-019-09815-7
Hu, L. Y., Blanc, G., & Noetinger, B. (2001). Gradual deformation and iterative calibration of sequential stochastic simulations. Mathematical Geology, 33(4), 475–489.
Jafarpour, B. (2011). Wavelet reconstruction of geologic facies from nonlinear dynamic flow measurements. IEEE Transactions on Geoscience and Remote Sensing, 49(5), 1520–1535. https://doi.org/10.1109/TGRS.2010.2089464
Jafarpour, B., Goyal, V. K., McLaughlin, D. B., & Freeman, W. T. (2009). Transform-domain sparsity regularization for inverse problems in geosciences. Geophysics, 74(5), R69–R83. https://doi.org/10.1190/1.3157250
Jiang, A., & Jafarpour, B. (2021). Deep convolutional autoencoders for robust flow model calibration under uncertainty in geologic continuity. Water Resources Research, 57, e2021WR029754. https://doi.org/10.1029/2021WR029754
Journel, A., & Zhang, T. (2007). The necessity of a multiple-point prior model. Mathematical Geology, 38(5), 591–610. https://doi.org/10.1007/s11004-006-9031-2
Kessler, T. C., Comunian, A., Oriani, F., Renard, P., Nilsson, B., Klint, K. E., et al. (2013). Modeling Fine-scale geological heterogeneity-examples of sand lenses in tills. Ground Water, 51(5), 692–705. https://doi.org/10.1111/j.1745-6584.2012.01015.x
Kessler, T. C., Klint, K. E. S., Nilsson, B., & Bjerg, P. L. (2012). Characterization of sand lenses embedded in tills. Quaternary Science Reviews, 53, 55–71. https://doi.org/10.1016/j.quascirev.2012.08.011
Khaninezhad, M. M., Jafarpour, B., & Li, L. (2012). Sparse geologic dictionaries for subsurface flow model calibration: Part I. Inversion formulation. Advances in Water Resources, 39, 106–121. https://doi.org/10.1016/j.advwatres.2011.09.002
Kingma, D. P., & Welling, M. (2014). Auto-encoding variational Bayes. ArXiv:1312.6114 [cs, stat]. Retrieved from http://arxiv.org/abs/1312.6114
Kleinberg, R., Li, Y., & Yuan, Y. (2018). An alternative view: When does SGD Escape local minima? arXiv:1802.06175 [cs]. Retrieved from https://arxiv.org/abs/1802.061752
Kramer, M. A. (1991). Nonlinear principal component analysis using autoassociative neural networks. AIChE Journal, 37(2), 233–243. https://doi.org/10.1002/aic.690370209
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90. https://doi.org/10.1145/3065386
Laloy, E., Hérault, R., Jacques, D., & Linde, N. (2018). Training-image based geostatistical inversion using a spatial generative adversarial neural network. Water Resources Research, 54, 381–406. https://doi.org/10.1002/2017WR022148
Laloy, E., Hérault, R., Lee, J., Jacques, D., & Linde, N. (2017). Inversion using a new low-dimensional representation of complex binary geological media based on a deep neural network. Advances in Water Resources, 110, 387–405. https://doi.org/10.1016/j.advwatres.2017.09.029
Laloy, E., Linde, N., Ruffino, C., Hérault, R., Gasso, G., & Jacques, D. (2019). Gradient-based deterministic inversion of geophysical data with generative adversarial networks: Is it feasible? Computers & Geosciences, 133, 104333. https://doi.org/10.1016/j.cageo.2019.104333
Larsen, T. H., Palstrøm, P., & Larsen, L. (2016). Kallerup Grusgrav: Kortlægning af sandlinser i moræneler (Tech. Rep. 3641400021). Orbicon A/S.
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., et al. (1989). Backpropagation applied to handwritten zip code recognition. Neural Computation, 1(4), 541–551. https://doi.org/10.1162/neco.1989.1.4.541
Lemmens, L., Rogiers, B., Jacques, D., Huysmans, M., Swennen, R., Urai, J. L., et al. (2019). Nested multiresolution hierarchical simulated annealing algorithm for porous media reconstruction. Physical Review E, 100(5), 053316. https://doi.org/10.1103/PhysRevE.100.053316
Linde, N., Renard, P., Mukerji, T., & Caers, J. (2015). Geological realism in hydrogeological and geophysical inverse modeling: A review. Advances in Water Resources, 86, 86–101. https://doi.org/10.1016/j.advwatres.2015.09.019
Looms, M. C., Klotzsche, A., van der Kruk, J., Larsen, T. H., Edsen, A., Tuxen, N., et al. (2018). Mapping sand layers in clayey till using crosshole ground-penetrating radar. Geophysics, 83(1), A21–A26. https://doi.org/10.1190/geo2017-0297.1
Looms, M. C., Klotzsche, A., van der Kruk, J., Larsen, T. H., Edsen, A., Tuxen, N., et al. (2021). Crosshole ground penetrating radar data collected in clayey till with sand occurrences. Pangaea. https://doi.org/10.1594/PANGAEA.934056
Lopez-Alvis, J., Hermans, T., & Nguyen, F. (2019). A cross-validation framework to extract data features for reducing structural uncertainty in subsurface heterogeneity. Advances in Water Resources, 133, 103427. https://doi.org/10.1016/j.advwatres.2019.103427
Lopez-Alvis, J., Laloy, E., Nguyen, F., & Hermans, T. (2021). Deep generative models in inversion: The impact of the generator’s nonlinearity and development of a new approach based on a variational autoencoder. Computers & Geosciences, 152, 104762. https://doi.org/10.1016/j.cageo.2021.104762
Lucic, M., Kurach, K., Michalski, M., Gelly, S., & Bousquet, O. (2018). Are GANs created equal? A large-scale study. arXiv:1711.10337 [cs, stat]. Retrieved from http://arxiv.org/abs/1711.10337
Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., & Frey, B. (2015). Adversarial autoencoders. ArXiv:1511.05644 [cs]. Retrieved from http://arxiv.org/abs/1511.05644
Mariethoz, G. (2018). When should we use multiple-point geostatistics? In B. Daya Sagar, Q. Cheng, & F. Agterberg (Eds.), Handbook of mathematical geosciences: Fifty years of IAMG (pp. 645–653). Springer International Publishing. https://doi.org/10.1007/978-3-319-78999-6_31
Mariethoz, G., Renard, P., & Straubhaar, J. (2010). The direct sampling method to perform multiple-point geostatistical simulations: Performing multiple-points simulations. Water Resources Research, 46, W11536. https://doi.org/10.1029/2008WR007621
Maurer, H., Holliger, K., & Boerner, D. (1998). Stochastic regularization: Smoothness or similarity. Geophysical Research Letters, 25(15), 2889–2892.
Mosser, L., Dubrule, O., & Blunt, M. J. (2018). Stochastic seismic waveform inversion using generative adversarial networks as a geological prior. arXiv:1806.03720 [physics, stat]. Retrieved from http://arxiv.org/abs/1806.03720
Oware, E. K., Irving, J., & Hermans, T. (2019). Basis-constrained Bayesian Markov-chain Monte Carlo difference inversion for geoelectrical monitoring of hydrogeologic processes. Geophysics, 84(4), A37–A42. https://doi.org/10.1190/geo2018-0643.1
Park, H., Scheidt, C., Fenwick, D., Boucher, A., & Caers, J. (2013). History matching and uncertainty quantification of facies models with multiple geological interpretations. Computational Geosciences, 17(4), 609–621. https://doi.org/10.1007/s10596-013-9343-5
Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., & Lerer, A. (2017). Automatic differentiation in PyTorch.
Radford, A., Metz, L., & Chintala, S. (2016). Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv:1511.06434 [cs]. Retrieved from http://arxiv.org/abs/1511.06434
Renard, P., & Allard, D. (2013). Connectivity metrics for subsurface flow and transport. Advances in Water Resources, 51, 168–196. https://doi.org/10.1016/j.advwatres.2011.12.001
Rezende, D. J., Mohamed, S., & Wierstra, D. (2014). Stochastic backpropagation and approximate inference in deep generative models. arXiv:1401.4082 [cs, stat]. Retrieved from http://arxiv.org/abs/1401.4082
Sajjadi, M. S. M., Bachem, O., Lucic, M., Bousquet, O., & Gelly, S. (2018). Assessing generative models via Precision and Recall. arXiv:1806.00035 [cs, stat]. Retrieved from http://arxiv.org/abs/1806.00035
Sarma, P., Durlofsky, L. J., & Aziz, K. (2008). Kernel principal component analysis for efficient, differentiable parameterization of multipoint geostatistics. Mathematical Geosciences, 40(1), 3–32. https://doi.org/10.1007/s11004-007-9131-7
Scheidt, C., Fernandes, A. M., Paola, C., & Caers, J. (2016). Quantifying natural delta variability using a multiple-point geostatistics prior uncertainty model: Delta variability and geostatistics. Journal of Geophysical Research: Earth Surface, 121, 1800–1818. https://doi.org/10.1002/2016JF003922
Scheidt, C., Li, L., & Caers, J. (2018). Quantifying uncertainty in subsurface systems (No. 236). John Wiley and Sons & American Geophysical Union.
Seo, J. K., Kim, K. C., Jargal, A., Lee, K., & Harrach, B. (2019). A learning-based method for solving ill-posed nonlinear inverse problems: A simulation study of lung EIT. SIAM Journal on Imaging Sciences, 12(3), 1275–1295. https://doi.org/10.1137/18M1222600
Silva, D. A., & Deutsch, C. V. (2012). Multiple point statistics with multiple training images. CCG Annual Report, 14, 8.
Soille, P. (2004). Morphological image analysis. Springer. https://doi.org/10.1007/978-3-662-05088-0
Straubhaar, J., Renard, P., Mariethoz, G., Froidevaux, R., & Besson, O. (2011). An improved parallel multiple-point Algorithm using a list approach. Mathematical Geosciences, 43(3), 305–328. https://doi.org/10.1007/s11004-011-9328-7
Strebelle, S. (2002). Conditional simulation of complex geological structures using multiple-point statistics. Mathematical Geology, 34(1), 1–21. https://doi.org/10.1023/A:1014009426274
Tarantola, A., & Valette, B. (1982). Inverse problems = quest for information. Journal of Geophysics, 50(3), 159–170.
Tikhonov, A. N., & Arsenin, V. I. A. (1977). Solutions of ill-posed problems. Winston and Sons.
Torquato, S., Beasley, J. D., & Chiew, Y. C. (1988). Two-point cluster function for continuum percolation. The Journal of Chemical Physics, 88(10), 6540–6547. https://doi.org/10.1063/1.454440
Treister, E., & Haber, E. (2016). A fast marching algorithm for the factored eikonal equation. Journal of Computational Physics, 324, 210–225. https://doi.org/10.1016/j.jcp.2016.08.012
Uria, B., Murray, I., & Larochelle, H. (2014). A deep and tractable density estimator. arXiv:1310.1757 [cs, stat]. Retrieved from http://arxiv.org/abs/1310.1757
van der Walt, S., Schönberger, J. L., Nunez-Iglesias, J., Boulogne, F., Warner, J. D., Yager, N., et al. (2014). scikit-image: Image processing in Python. PeerJ, 2, e453. https://doi.org/10.7717/peerj.453
Vo, H. X., & Durlofsky, L. J. (2014). A new differentiable parameterization based on principal component analysis for the low-dimensional representation of complex geological models. Mathematical Geosciences, 46(7), 775–813. https://doi.org/10.1007/s11004-014-9541-2
Yacoby, Y., Pan, W., & Doshi-Velez, F. (2021). Failure modes of variational autoencoders and their effects on downstream tasks. (_eprint: 2007.07124).
You, N., Li, Y. E., & Cheng, A. (2021). 3D Carbonate digital rock reconstruction using progressive growing GAN. Journal of Geophysical Research: Solid Earth, 126, e2021JB021687. https://doi.org/10.1029/2021JB021687
Zhang, C., Butepage, J., Kjellstrom, H., & Mandt, S. (2018). Advances in variational inference. ArXiv:1711.05597 [cs, stat]. Retrieved from http://arxiv.org/abs/1711.05597
