DINCAE: multivariate convolutional neural network with error estimates to reconstruct gridded and along-track satellite observations

DINCAE (Data INterpolating Convolutional Auto-Encoder) is a neural network to reconstruct missing data (e.g. obscured by clouds) in satellite data. The technique has been described in Barth et al. (2020, https://doi.org/10.5194/gmd-13-1609-2020) for a single variable. Contrary to standard image reconstruction with neural networks, this application requires a method to handle missing data (or data with variable accuracy) in the training phase. Instead of using a standard L2 (or L1) cost function, the neural network (U-net type of network) is optimized by minimizing the negative log likelihood assuming a Gaussian distribution (characterized by a mean and a variance). As a consequence, the neural network also provides an expected error variance of the reconstructed field (per pixel and per time instance).The method has been extended to handle multivariate data (an example will be shown with sea-surface temperature, chlorophyll and wind fields) and the structure of the neural network has been updated. The improvement of this network is demonstrated in the Adriatic Sea.
The code has been ported from Python TensorFlow 1.15 to Julia with Knet.jl which reduces the training time from 3.5 to 1.9 hours for the same network architecture. The speed-up is primarily thanks to a more efficient data transformation which is used to expand the training dataset by data augmentation. The first convolutional layers and the cost function have been modified so that also unstructured data can be used as inputs to obtain gridded fields as output. To demonstrate this, the neural network is applied to along-track altimetry data in the Mediterranean Sea. Results from a 20-year reconstruction are presented and validated. Hyperparameters are determined using Bayesian optimization and minimizing the error relative to a development dataset.