Cultural heritage; Image processing; Infrared thermography; Machine learning; Non-destructive testing; Painting; Terahertz; Background segmentation; Cultural heritages; Gaussian modeling; Images processing; Machine-learning; Mixture of Gaussians; Non destructive testing; Segmentation algorithms; Tera Hertz; Terahertz time-domain spectroscopy; Mechanics of Materials; Mechanical Engineering
Abstract :
[en] With increasing attention paid to the protection of cultural relics, non-destructive testing (NDT) technologies are thought to be profoundly rewarding, resulting in a widespread uptake of feature-extraction algorithms and defect-detection techniques. Among various alternatives, infrared thermography (IRT) and Terahertz time-domain spectroscopy (THz-TDS) are non-invasive in nature and thus are appropriate for applications involving ancient buildings and artworks. The present study is motivated by the fact that online/offline background segmentation algorithms based on the Gaussian mixture model and widely used in video processing to distinguish between foreground and background, can be successfully integrated as a feature extraction tool with NDT. Since IRT and THz-TDS image sequences resemble a video, the image length and width can be taken as the first two dimensions, and time can serve as the third one. Such sequences can be processed effectively by the background segmentation algorithms, thus can help detect defects of different types at varying depths. The experimental section of the paper considers a tempera painting (a replica of Botticelli’s “The Birth of Venus”) with artificiallyintroduced defects. For benchmarking purposes, the background segmentation algorithms (based on a mixture of Gaussian models) are compared with the fast Fourier transform and principal component analysis to demonstrate the superior performance of the proposed novel algorithms.
Research Center/Unit :
CSL - Centre Spatial de Liège - ULiège
Disciplines :
Physics
Author, co-author :
Li, Qi; School of Automation Science and Engineering, South China University of Technology, Guangzhou, China ; Computer Vision and Systems Laboratory (CVSL), Department of Electrical and Computer Engineering, Laval University, Quebec, Canada
Zhang, Hai ; Centre for Composite Materials and Structures (CCMS), Harbin Institute of Technology, Harbin, China ; Computer Vision and Systems Laboratory (CVSL), Department of Electrical and Computer Engineering, Laval University, Quebec, Canada
Hu, Jue; Centre for Composite Materials and Structures (CCMS), Harbin Institute of Technology, Harbin, China
Sfarra, Stefano; Department of Industrial and Information Engineering and Economics, University of L’Aquila, L’Aquila, Italy
Mostacci, Miranda; Professional Restorer, Celano, Italy
Yang, Dazhi; School of Electrical Engineering and Automation, Harbin Institute of Technology, Harbin, China
Georges, Marc ; Université de Liège - ULiège > Centres généraux > CSL (Centre Spatial de Liège)
Vavilov, Vladimir P.; Tomsk Polytechnic University, Tomsk, Russian Federation
Maldague, Xavier P. V.; Computer Vision and Systems Laboratory (CVSL), Department of Electrical and Computer Engineering, Laval University, Quebec, Canada
Language :
English
Title :
Using the Unsupervised Mixture of Gaussian Models for Multispectral Non-destructive Evaluation of the Replica of Botticelli’s “The Birth of Venus”
Mitacs MRIF - Gouvernement du Québec. Ministère des Relations internationales et de la Francophonie NSERC - Natural Sciences and Engineering Research Council CRCs - Canada Research Chairs
Funding text :
Qi Li acknowledges the Mitacs Globalink Research Internship Program under Award No.113342 and the China Scholarship Council (CSC) under award No.202106155015.This work is supported by the Canada Research Chair in Multipolar Infrared Vision (MiViM), the Natural Sciences and Engineering Research Council (NSERC) of Canada through the Discovery Grant program and the Create-oN DuTy! Program. This work is also supported by the Ministère des Relations internationales et de la Francophonie (MRIF) through the 11th Coopération Québec-Wallonie-Bruxelles project «11.812 Contrôle NonDestructif par Imagerie Numérique et TeraHertz (CINTera)» and Mitacs [Globalink Research Internship Program under Award No. 113342].
Schumann, W., Zürcher, J.P., Cuche, D.: Holography and Deformation Analysis. Springer, Berlin, Heidelberg (1985). 10.1007/978-3-540-38981-1_1 DOI: 10.1007/978-3-540-38981-1_1
Vest, C.M.: Holographic Interferometry. China Machine Press, Beijing (1984). (in Chinese)
Sfarra, S., Ibarra-Castanedo, C., Ambrosini, D., Paoletti, D., Bendada, A., Maldague, X.: Discovering the defects in paintings using non-destructive testing (NDT) techniques and passing through measurements of deformation. J. Nondestr. Eval. 33, 358–383 (2014). 10.1007/s10921-013-0223-7 DOI: 10.1007/s10921-013-0223-7
Vincitorio, F., Ramil, A., López, A. J., Saavedra, E., Yáñez, A.: NDT testing by holographic interferometry applied to the structural diagnostic of artwork conservations by means of sound wave excitation. In: Eighth International Conference on Vibration Measurements by Laser Techniques: Advances and Applications, pp. 184–191 (2008). https://doi.org/10.1117/12.802991
Li, X.: Wavelet transform for detection of partial fringe patterns induced by defects in nondestructive testing of holographic interferometry and electronic speckle pattern interferometry. Opt. Eng. 39, 2821–2827 (2000). 10.1117/1.1308485 DOI: 10.1117/1.1308485
Markov, V.B., Boone, P.M., Burykin, N.M., Ovsyannikov, V.V.: Defect localization in museum items with traditional and electronic holographic nondestructive testing. In: International Conference on Optical Diagnostics of Materials and Devices for Opto-, Micro-, and Quantum Electronics, pp. 548–556 (1995). https://doi.org/10.1117/12.226224
Alvarez-Arenas, T.G.: Acoustic impedance matching of piezoelectric transducers to the air. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 51, 624–633 (2004). 10.1109/TUFFC.2004.1320834 DOI: 10.1109/TUFFC.2004.1320834
Shanmugam, P., Iglesias, L., Michaud, J.F., Alquier, D., Colin, L., Dufour, I., Certon, D.: Broad bandwidth air-coupled micromachined ultrasonic transducers for gas sensing. Ultrasonics 114, 106410 (2021). 10.1016/j.ultras.2021.106410 DOI: 10.1016/j.ultras.2021.106410
Chang, J.J., Chen, Z.H.: Development and application of high signal-to-noise ratio air-coupled ultrasonic sensorin. 2019 Far East NDT New Technology & Application Forum (FENDT), pp. 173–177 (2019). https://doi.org/10.1109/FENDT47723.2019.8962777
Minkina, W.: Theoretical basics of radiant heat transfer–practical examples of calculation for the infrared (IR) used in infrared thermography measurements. Quant. InfraRed Thermogr. J. 18, 269–282 (2021). 10.1080/17686733.2020.1738164 DOI: 10.1080/17686733.2020.1738164
Vavilov, V.P., Karabutov, A.A., Chulkov, A.O., Derusova, D.A., Moskovchenko, A.I., Cherepetskaya, E.B., Mironova, E.A.: Comparative study of active infrared thermography, ultrasonic laser vibrometry and laser ultrasonics in application to the inspection of graphite/epoxy composite parts. Quant. InfraRed Thermogr. J. 17, 235–248 (2020). 10.1080/17686733.2019.1646971 DOI: 10.1080/17686733.2019.1646971
Zhang, H., Sfarra, S., Saluja, K., Peeters, J., Fleuret, J., Duan, Y., Fernandes, H., Avdelidis, N., Ibarra-Castanedo, C., Maldague, X.: Non-destructive investigation of paintings on canvas by continuous wave terahertz imaging and flash thermography. J. Nondestr. Eval. 36, 1–12 (2017). 10.1007/s10921-017-0414-8 DOI: 10.1007/s10921-017-0414-8
Hu, J., Zhang, H., Sfarra, S., Gargiulo, G., Avdelidis, N.P., Zhang, M., Yang, D., Maldague, X.: Non-destructive imaging of marqueteries based on a new infrared-terahertz fusion technique. Infrared Phys. Technol. (2022). 10.1016/j.infrared.2022.104277 DOI: 10.1016/j.infrared.2022.104277
Hu, J., Zhang, H., Sfarra, S., Pivarčiová, E., Yao, Y., Duan, Y., Ibarra-Castanedo, C., Tian, G., Maldague, X.: Autonomous dynamic line-scan continuous-wave terahertz non-destructive inspection system combined with unsupervised exposure fusion. NDT E Int. 132, 102705 (2022). 10.1016/j.ndteint.2022.102705 DOI: 10.1016/j.ndteint.2022.102705
Cooley, J.W., Lewis, P.A.W., Welch, P.D.: The fast Fourier transform and its applications. IEEE Trans. Educ. 12, 27–34 (1969). 10.1109/TE.1969.4320436 DOI: 10.1109/TE.1969.4320436
Rajic, N.: Principal component thermography for flaw contrast enhancement and flaw depth characterisation in composite structures. Compos. Struct. 58, 521–528 (2002). 10.1016/S0263-8223(02)00161-7 DOI: 10.1016/S0263-8223(02)00161-7
Ghaderi, H., Kabiri, P.: Automobile independent fault detection based on acoustic emission using FFT. In: Singapore International NDT Conference & Exhibition (SINCE 2011) (2011). https://www.ndt.net/article/SINCE2011/papers/11_Kabiri_Rev2.pdf
Hanbay, K., Talu, M.F., Özgüven, Ö.F.: Fabric defect detection systems and methods—a systematic literature review. Optik 127, 11960–11973 (2016). 10.1016/j.ijleo.2016.09.110 DOI: 10.1016/j.ijleo.2016.09.110
Marcomini, L.A., Cunha, A.L.: A comparison between background modelling methods for vehicle segmentation in highway traffic videos. arXiv Preprint. (2018). https://doi.org/10.48550/arXiv.1810.02835
Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149), pp. 246–252 (1999). https://doi.org/10.1109/CVPR.1999.784637
Allili, M.S., Bouguila, N., Ziou, D.: A robust video foreground segmentation by using generalized Gaussian mixture modeling. In: Fourth Canadian Conference on Computer and Robot Vision (CRV’07), pp. 503–509 (2007). https://doi.org/10.1109/CRV.2007.7
Meng, D., De La Torre, F.: Robust matrix factorization with unknown noise. In: 2013 IEEE International Conference on Computer Vision, pp. 1337–1344 (2013). https://doi.org/10.1109/ICCV.2013.169
Yong, H., Meng, D., Zuo, W., Zhang, L.: Robust online matrix factorization for dynamic background subtraction. IEEE Trans. Pattern Anal. Mach. Intell. 40, 1726–1740 (2017). 10.1109/TPAMI.2017.2732350 DOI: 10.1109/TPAMI.2017.2732350
Botticelli, S.: The Birth of Venus. https://www.uffizi.it/en/artworks/birth-of-venus. Accessed 9 Sept 2022
Laureti, S., Malekmohammadi, H., Rizwan, M.K., Burrascano, P., Sfarra, S., Mostacci, M., Ricci, M.: Looking through paintings by combining hyper-spectral imaging and pulse-compression thermography. Sensors 19, 4335 (2019). 10.3390/s19194335 DOI: 10.3390/s19194335
Cielo, P.: Pulsed photothermal evaluation of layered materials. J. Appl. Phys. 56, 230–234 (1984). 10.1063/1.333758 DOI: 10.1063/1.333758
Bozzoli, F., Cattani, L., Mocerino, A., Rainieri, S., Tougri, I., Colaço, M.J.: Characterisation of the heat transfer in displaced enhancement devices by means of inverse problem approach applied to IR images. Quant. InfraRed Thermogr. J. 18, 108–126 (2021). 10.1080/17686733.2019.1700696 DOI: 10.1080/17686733.2019.1700696
Sakagami, T., Kubo, S.: Applications of pulse heating thermography and lock-in thermography to quantitative nondestructive evaluations. Infrared Phys. Technol. 43, 211–218 (2002). 10.1016/S1350-4495(02)00141-X DOI: 10.1016/S1350-4495(02)00141-X
Maldague, X., Krapez, J.C., Cielo, P.: Subsurface flaw detection in reflective materials by thermal-transfer imaging. In: Thermosense XI: International Conference on Thermal Infrared Sensing for Diagnostics and Control, vol. 1094, pp. 163–174 (1989). https://doi.org/10.1117/12.953399
Larbi Youcef, M., Feuillet, V., Ibos, L., Candau, Y.: In situ quantitative diagnosis of insulated building walls using passive infrared thermography. Quant. InfraRed Thermogr. J. 19, 41–69 (2022). 10.1080/17686733.2020.1805939 DOI: 10.1080/17686733.2020.1805939
Tao, N., Wang, C., Zhang, C., Sun, J.: Quantitative measurement of cast metal relics by pulsed thermal imaging. Quant. InfraRed Thermogr. J. 19, 27–40 (2022). 10.1080/17686733.2020.1799304 DOI: 10.1080/17686733.2020.1799304
Liu, K., Huang, K.L., Sfarra, S., Yang, J., Liu, Y., Yao, Y.: Factor analysis thermography for defect detection of panel paintings. Quant. InfraRed Thermogr. J. (2022). 10.1080/17686733.2021.2019658 DOI: 10.1080/17686733.2021.2019658
Serantoni, V., Jourdan, F., Louche, H., Sultan, A.: Proposal for a protocol using an infrared microbolometer camera and wavelet analysis to study foot thermoregulation. Quant. InfraRed Thermogr. J. 18, 73–91 (2021). 10.1080/17686733.2019.1697847 DOI: 10.1080/17686733.2019.1697847
Shoa, P., Hemmat, A., Gheysari, M., Amirfattahi, R.: Effect of micro climatic indices on the accuracy of thermographic plant water status monitoring, case study of a semi-arid area. Quant. InfraRed Thermogr. J. 18, 283–299 (2021). 10.1080/17686733.2020.1768496 DOI: 10.1080/17686733.2020.1768496
Zhong, S.: Progress in terahertz nondestructive testing: a review. Front. Mech. Eng. 14, 273–281 (2019). 10.1007/s11465-018-0495-9 DOI: 10.1007/s11465-018-0495-9
Redo-Sanchez, A., Laman, N., Schulkin, B., Tongue, T.: Review of terahertz technology readiness assessment and applications. J. Infrared Millim. Terahertz Waves 34, 500–518 (2013). 10.1007/s10762-013-9998-y DOI: 10.1007/s10762-013-9998-y
Guo, J.: Study on terahertz time-domain spectral imaging. Dissertation, Changchun University of Science and Technology (2019)
Moon, T.K.: The expectation-maximization algorithm. IEEE Signal Process. Mag. 13, 47–60 (1996). 10.1109/79.543975 DOI: 10.1109/79.543975
Cao, X., Zhao, Q., Meng, D., Chen, Y., Xu, Z.: Robust low-rank matrix factorization under general mixture noise distributions. IEEE Trans. Image Process. 25, 4677–4690 (2016). 10.1109/TIP.2016.2593343 DOI: 10.1109/TIP.2016.2593343
Wang, X., Gou, S., Wang, X., Zhao, Y., Zhang, L.: Patch-based gaussian mixture model for concealed object detection in millimeter-wave images. In: TENCON 2018-2018 IEEE Region 10 Conference, pp. 2522–2527 (2018). https://doi.org/10.1109/TENCON.2018.8650148
Kroonenberg, P.M., De Leeuw, J.: Principal component analysis of three-mode data by means of alternating least squares algorithms. Psychometrika 45, 69–97 (1980). 10.1007/BF02293599 DOI: 10.1007/BF02293599
Gauvain, J.L., Lee, C.H.: Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains. IEEE Trans. Speech Audio Process. 2, 291–298 (1994). 10.1109/89.279278 DOI: 10.1109/89.279278
Ibarra-Castanedo, C., Maldague, X.P.: Handbook of Technical Diagnostics, pp. 175–220. Springer, Berlin, Heidelberg (2013). 10.1007/978-3-642-25850-3_10 DOI: 10.1007/978-3-642-25850-3_10
