Mutual information based registration of multi- and hyperspectral imaging data applied to paintings by Théo Van Rysselberghe

Multimodal and hyperspectral imaging techniques are now widely used in the technical study of paintings, as they provide crucial information on materials, execution techniques, and hidden features through non-invasive analysis. However, combining these different datasets remains challenging due to differences in spatial resolution, local distortions, data volume and the often low radiometric correspondence between imaging modalities. This work presents a fully automated registration pipeline based on mutual information (MI) for the registration of multi- and hyperspectral images to a common reference image. The proposed method uses an affine transformation for coarse alignment, followed by a non-rigid B-spline transformation to correct local deformations. Both steps are optimised using an L-BFGS-B algorithm. To improve robustness, a multi-resolution and multi-start strategy was employed. The method was applied to multimodal imaging data acquired from four paintings by the Belgian painter Théo Van Rysselberghe, including high-resolution photography under visible and UV light, infrared reflectography, X-ray radiography, visible–near-infrared hyperspectral imaging, and macroscopic X-ray fluorescence spectroscopy. Several preprocessing strategies were evaluated and the results showed that no single configuration performed optimally across all modalities. In some cases, mild Gaussian blurring improved both mutual information values and visual alignment, whereas contrast-limited adaptive histogram equalisation often introduced noise and degraded performance. Registration accuracy was assessed using a normalised Target Registration Error computed from manually selected landmarks. Overall, the proposed method provided precise alignment of most imaging modalities commonly used in the study of paintings, including hyperspectral and elemental mapping data. The results also highlight the limitations of mutual information for modalities with low statistical dependence, such as transmitted-light photography and X-ray radiography.