A novel semi-distance for measuring dissimilarities of curves with sharp local patterns

With this article, we define, investigate and exploit an efficient measure of the dissimilarity between curves that show sharp local features. Examples for such data typically arise in numerous scientific fields, including medicine (e.g. H-NMR spectroscopic data for metabonomic analyses, EEG or ECG spectral analysis), geophysics (e.g. earth quake data) or astronomy (e.g. solar irradiance time series). A given peak in a set of such curves might be affected, from one curve to the other, by a vertical amplification, a horizontal shift or both simultaneously. Then, in the presence of horizontal shifts, commonly used dissimilarity measures do not return coherent results when comparing a large number of these curves, for instance for subsequent functional classification or prediction purposes. In this work we propose therefore a new dissimilarity measure which has the ability to capture both horizontal and vertical variations of the peaks, in a unified framework, i.e. in a coherent way within an integrated procedure (avoiding any preprocessing, e.g. in case of misalignment). This dissimilarity measure is embedded within a complete algorithmic procedure, which we call the Bagidis methodology, and which as such is our new proposal for investigating datasets of curves with sharp local features. We strongly suggest to use it replacing classical distances, such as the Euclidean distance between the values (vertical amplitudes) of the observed curve data, in any distance based statistical tool aimed at analyzing datasets with curves having sharp local patterns. Along some typical examples of curve comparison, e.g. in the context of classification or prediction, we show in particular how the use of the Bagidis distance improves the statistical analysis in many situations without being harmful for cases when not giving any advantage over the Euclidean distance (i.e. in the absence of horizontally shifted sharp local patterns). As a key ingredient of our approach we note that it is based upon the expansion of each curve in a different (orthogonal) wavelet basis, one that is particularly suited to the curve. In order to define the Bagidis (semi-) distance, we do not only take into account the differences between the projections of the series onto the bases, as usual, but also the differences between the bases. Therefore, the name Bagidis chosen for the method stands for BAses GIving DIStances.