Automated Identification of Blockx Paints through VNIR–XRF Spectral Data Fusion

The identification of pigments in historical paintings is essential for understanding
artistic practice, informing conservation decisions, and detecting later interventions.
Visible–near-infrared (VNIR) hyperspectral imaging and macro-X-ray fluorescence
(MA-XRF) spectroscopy are two complementary, non-invasive techniques widely used
for this purpose: VNIR captures the spectral reflectance signature of paintings, while
MA-XRF maps their elemental composition.
This work presents a pipeline for automated pigment classification that combines
registered VNIR and MA-XRF data through machine learning. The two hyperspectral
datacubes are first brought into a common spatial reference using a mutual information-based registration method consisting of an affine coarse alignment
followed by a B-spline non-rigid refinement. Once co-registered, each pixel is
described by both its spectral reflectance profile and its elemental signature, enabling
multimodal classification.
Classifiers are trained on reference colour charts manufactured by Blockx, a Belgian
oil paint maker. A well-preserved rectangular chart dating from 1936 serves as training
dataset, while an older colour chart from 1909 from the same manufacturer is used for
validation. While both colour charts are from the same manufacturer, aging and
conservation conditions are sources of data variation.
Several classifiers are compared including support vector machines, random forests,
and gradient boosting using both individual modalities and their fusion. Fusion
strategies range from early concatenation of feature vectors to calibrated probability weighted combination. First results indicate that the VNIR modality carries the primary
discriminative signal, while XRF features improve separation of chemically distinct but
spectrally similar pigments. Certain classes, notably cadmium-based yellows, exhibit
different VNIR spectral signatures due to conservation conditions and are thus prone
to misclassifications